Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
2000-05-26
2003-11-18
Rao, Andy (Department: 2713)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
C375S240090
Reexamination Certificate
active
06650705
ABSTRACT:
FIELD OF THE INVENTION
This invention relates generally to encoding and transcoding multiple video objects, and more particularly to a system that controls the encoding and transcoding of multiple video objects with variable temporal resolutions.
BACKGROUND OF THE INVENTION
Recently, a number of standards have been developed for communicating encoded information. For video sequences, the most widely used standards include MPEG-1 (for storage and retrieval of moving pictures), MPEG-2 (for digital television) and H.263, see ISO/IEC JTC1 CD 11172, MPEG, “Information Technology—Coding of Moving Pictures and Associated Audio for Digital Storage Media up to about 1.5 Mbit/s—Part 2: Coding of Moving Pictures Information,” 1991, LeGall, “MPEG: A Video Compression Standard for Multimedia Applications,” Communications of the ACM, Vol. 34, No. 4, pp. 46-58, 1991, ISO/IEC DIS 13818-2, MPEG-2, “Information Technology—Generic Coding of Moving Pictures and Associated Audio Information—Part 2: Video,” 1994, ITU-T SG XV, DRAFT H.263, “Video Coding for Low Bitrate Communication,” 1996, ITU-T SG XVI, DRAFT13 H.263+Q15-A-60 rev.0, “Video Coding for Low Bitrate Communication,” 1997.
These standards are relatively low-level specifications that primarily deal with the spatial and temporal compression of video sequences. As a common feature, these standards perform compression on a per frame basis. With these standards, one can achieve high compression ratios for a wide range of applications.
Newer video coding standards, such as MPEG-4 (for multimedia applications), see“Information Technology—Generic coding of audio/visual objects,” ISO/IEC FDIS 14496-2 (MPEG4 Visual), Nov. 1998, allow arbitrary-shaped objects to be encoded and decoded as separate video object planes (VOP). The objects can be visual, audio, natural, synthetic, primitive, compound, or combinations thereof. Video objects are composed to form compound objects or “scenes.”
The emerging MPEG-4 standard is intended to enable multimedia applications, such as interactive video, where natural and synthetic materials are integrated, and where access is universal. MPEG-4 allows for content based interactivity. For example, one might want to “cut-and-paste” a moving figure or object from one video to another. In this type of application, it is assumed that the objects in the multimedia content have been identified through some type of segmentation process, see for example, U.S. patent application Ser. No. 09/326,750 “Method for Ordering Image Spaces to Search for Object Surfaces” filed on Jun. 4, 1999 by Lin et al.
In the context of video transmission, these compression standards are needed to reduce the amount of bandwidth (available bit rate) that is required by the network. The network can represent a wireless channel or the Internet. In any case, the network has limited capacity and a contention for its resources must be resolved when the content needs to be transmitted.
Over the years, a great deal of effort has been placed on architectures and processes that enable devices to transmit the video content robustly and to adapt the quality of the content to the available network resources. Rate control is used to allocate the number of bits per coding time instant. Rate control ensures that the bitstream produced by an encoder satisfies buffer constraints.
Rate control processes attempt to maximize the quality of the encoded signal, while providing a constant bit rate. For frame-based encoding, such as MPEG-2, see U.S. Pat. No. 5,847,761, “Method for performing rate control in a video encoder which provides a bit budget for each frame while employing virtual buffers and virtual buffer verifiers,” issued to Uz, et al. on Dec. 8, 1998. For object-based encoding, such as MPEG-4, see U.S. Pat. No. 5,969,764, “Adaptive video coding method,” issued to Sun and Vetro on Oct. 19, 1999.
When the content has already been encoded, it is sometimes necessary to further convert the already compressed bitstream before the stream is transmitted through the network to accommodate, for example, a reduction in the available bit rate. Bit stream conversion or“transcoding” can be classified as bit rate conversion, resolution conversion, and syntax conversion. Bit rate conversion includes bit rate scaling and conversion between a constant bit rate (CBR) and a variable bit rate (VBR). The basic function of bit rate scaling is to accept an input bitstream and produce a scaled output bitstream that meets new load constraints of a receiver. A bit stream scaler is a transcoder, or filter, that provides a match between a source bitstream and the receiving load.
As shown in
FIG. 1
, typically, scaling can be accomplished by a transcoder
100
. In a brute force case, the transcoder includes a decoder
110
and encoder
120
. A compressed input bitstream
101
is fully decoded at an input rate Rin, then encoded at a new output rate Rout
102
to produce the output bitstream
103
. Usually, the output rate is lower than the input rate. However, in practice, full decoding and full encoding in a transcoder is not done due to the high complexity of encoding the decoded bitstream, instead the transcoding is done on a compressed or partial decoded bitstream.
Earlier work on MPEG-2 transcoding has been published by Sun et al., in “Architectures for MPEG compressed bitstream scaling,” IEEE Transactions on Circuits and Systems for Video Technology, April 1996. There, four methods of rate reduction, with varying complexity and architecture, were presented.
FIG. 2
shows an example method. In this architecture, the video bitstream is only partially decoded. More specifically, macroblocks of the input bitstream
201
are variable-length decoded (VLD)
210
. The input bitstream is also delayed
220
and inverse quantized (IQ)
230
to yield discrete cosine transform (DCT) coefficients. Given the desired output bit rate, the partially decoded data are analyzed
240
and a new set of quantizers is applied at
250
to the DCT macroblocks. These re-quantized macroblocks are then variable-length coded (VLC)
260
and a new output bitstream
203
at a lower rate can be formed. This scheme is much simpler than the scheme shown in
FIG. 1
because the motion vectors are re-used and an inverse DCT operation is not needed.
More recent work by Assuncao et al., in “A frequency domain video transcoder for dynamic bit-rate reduction of MPEG-2 bitstreams,” IEEE Transactions on Circuits and Systems for Video Technology, pp. 953-957, December 1998, describe a simplified architecture for the same task. They use a motion compensation (MC) loop, operating in the frequency domain for drift compensation. Approximate matrices are derived for fast computation of the MC macroblocks in the frequency domain. A Lagrangian optimization is used to calculate the best quantizer scales for transcoding.
Other work by Sorial et al, “Joint transcoding of multiple MPEG video bitstreams,” Proceedings of the International Symposium on Circuits and Systems, May 1999, presents a method of jointly transcoding multiple MPEG-2 bitstreams, see also U.S. patent application Ser. No. 09/410,552 “Estimating Rate-Distortion Characteristics of Binary Shape Data,” filed Oct. 1, 1999 by Vetro et al.
According to prior art compression standards, the number of bits allocated for encoding texture information is controlled by a quantization parameter (QP). The above papers are similar. Changing the QP on the basis of information contained in the original bitstream reduces the rate of texture bits. For an efficient implementation, the information is usually extracted directly in the compressed domain and can include measures that relate to the motion of macroblocks or residual energy of DCT macroblocks. This type of analysis can be found in the bit allocation analyzer
240
of FIG.
2
.
In addition to the above classical methods of transcoding, some new methods of transcoding have been described, for example, see U.S. patent application Ser. No. 09/504,323 “Object-Based Bitstream Transcoder,” filed by Vetro et al. on Feb. 14, 2000, for
Sun Huifang
Vetro Anthony
Brinkman Dirk
Curtin Andrew
Mitsubishi Electric Research Laboratories Inc.
Rao Andy
LandOfFree
Method for encoding and transcoding multiple video objects... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method for encoding and transcoding multiple video objects..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method for encoding and transcoding multiple video objects... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3148733