Television – Synchronization – Locking of video or audio to reference timebase
Reexamination Certificate
2000-04-04
2004-05-25
Lee, Michael H. (Department: 2614)
Television
Synchronization
Locking of video or audio to reference timebase
C348S425300, C348S425400, C348S722000
Reexamination Certificate
active
06741290
ABSTRACT:
The present invention relate to processing of coded video signals, particularly but not exclusively MPEG format compressed video. The invention is particularly, but not exclusively, concerned with real-time editing of video signals.
A problem with transmitting or storing video signals as a compressed or coded video data stream is that it is not normally possible for a decoder to commence decoding a new data stream instantaneously, but there is normally a delay of several frames while the decoder establishes synchronisation. For example with an MPEG data stream, a video sequence is sent as a series of groups of pictures (“GOP”), each GOP comprising an intra-coded frame (“I frame”) and one or more frames predicted from the I frame, either by forward prediction (“P frames”) or by bi-directional prediction (“B frames”). A decoder cannot commence decoding of the sequence until it receives an I frame. In addition, frames are buffered and processed within a decoder, so there can be an appreciable and unpredictable delay between supply of a video data stream to a decoder and output of the sequence of video frames.
For practical purposes, this presents a problem only when the decoder first receives a new signal; once synchronised, the decoder should continue to produce an output reliably. However, difficulties are encountered when it is desired to switch from one coded video source to another. In particular, a decoder buffer can overflow or underflow if the data stream input to the decoder is simply switched from one source to another.
The problems associated with switching between coded video sources have been recognised, and techniques have been proposed for switching between compressed bitstreams with minimal output degradation at the point of switching. An example of such a technique is described in our International Application No. WO 97/08898.
Prior art techniques have generally concentrated on dealing with problems occurring at or about the point of switching, and generally aim to provide a “good compromise” solution.
In general terms, the inventor has proposed that coded video source material is modified, if necessary, to adjust timing of output of frames from a downstream decoder, and preferably to establish and maintain a desired synchronisation of a downstream decoder. This can be achieved by setting the timing of the video source material (for example by explicitly adjusting values of timing fields contained in the data stream, or by altering the time at which the data is actually output to the decoder, or both) based on a studio reference clock. This can also (additionally or alternatively) be achieved by inserting “synthetic” frames (frames which have been generated and added to the original video source material) within the output data stream, so that a particular frame is displayed at a particular time. It is noted that if decoder synchronization is maintained in this way, the timing of display of particular frames need not be explicitly specified.
Looked at from another point of view, with the invention, the timing information contained in video source material (this is normally included to enable reliable playback of that material as intended) may be deliberately altered or additional frames may be added. Conventionally, it is not generally considered desirable to tamper with the internal, self-consistent, timing information contained in coded video data. However, this novel basic technique may provide numerous advantages, as will become clear as the description proceeds. Surprisingly, although the invention is in its most general terms not directed specifically at the switching of bitstreams, one advantage is that switching from one video source to another can be facilitated if both sources are synchronised. Another advantage is that playout of coded video material may be more readily controlled, for example for starting or pausing at a particular time.
According to a first, aspect, the invention provides apparatus for outputting a coded video sequence comprising:
means for receiving a coded video sequence to be output;
means for outputting the coded video sequence in a form enabling real-time decoding of the sequence to produce a sequence of pictures;
means for processing the coded video sequence and/or adjusting the timing of said outputting to produce decoded picture output from a decoder receiving the output sequence at a selected timing.
In this way, contrary to conventional compressed video sources, the timing information contained within the original sequence, or the actual time of output can be controlled deliberately based on external criteria, rather than in dependence on the timing data contained with the video sequence. This facilitates synchronisation of multiple video sources.
The video input sequence will typically be received from storage such as a disk, for example as a computer data file, and may be sent over a computer network. However, the sequence may be received in real-time, for example off-air, and may be contained in a multiple transport stream containing several sequences and other data. Where the data is received in real-time, the input data will usually need to be buffered, to enable differences in timing between input and output, and, of course, output cannot occur before the data has been received. As will be appreciated, as the size of the buffer increases, the distinction between receipt of “real-time” data and access of stored sequences diminishes.
Preferably, the apparatus includes means for receiving a studio reference clock, the processing and/or the timing of the outputting being controlled in dependence on timing information derived from the studio reference clock. This feature may facilitate synchronisation with other apparatus within a studio.
The studio reference clock is preferably supplied as a relatively high frequency clock counter having a relatively large maximum value, preferably at least 50 kHz and at least about 32 binary bits, “wide” (by which is meant the information content; the clock value may be supplied in another format, such as Binary Coded Decimal, in which case about 9 or 10 nibbles or digits would be required). In a preferred practical implementation, the clock is in a format that is readily or directly convertable to the format of timing fields within the video sequence. For example, the studio clock may be supplied directly as MPEG PCR values.
The, apparatus may include means for determining an effective clock frequency for the output coded video sequence based on the rate of change of clock values (for example PCR fields within an MPEG video sequence) contained within the output data and means for comparing or synchronising the effective clock frequency to the studio reference clock frequency. This may enable errors in original timing information to be identified and corrected, to achieve playback at a precisely controlled rate.
The apparatus may also include means for inputting information specifying a desired picture output time of at least one frame or field of a video sequence, said processing and/or adjusting being selected based on the desired picture output time and the studio reference clock so that said at least one frame or field is output from a decoder at the desired picture output time. In this way, the apparatus is able to look ahead, and prepare a video sequence for output at a precisely defined time, which solves a common problem in playing back video sequences live. It has been appreciated that solutions to problems of run-up or lead-in commonly employed in analogue storage systems (recording a 5 second lead-in sequence prior to The first frame required and commencing playing of the sequence 5 seconds before the frame is required) cannot be reliably used with coded sequences, as there is a somewhat more unpredictable delay before a decoder establishes synchronisation.
One way in which picture output time can be set is by altering one or more values in timing fields contained within the video sequence (for example PCR (programme clock reference) or PTS (presentation time stamp) or DTS (d
British Broadcasting Corporation
Lee Michael H.
McDermott & Will & Emery
LandOfFree
Processing coded video does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Processing coded video, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Processing coded video will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3230493