Image analysis – Image compression or coding – Interframe coding
Reexamination Certificate
1999-03-03
2004-02-10
Johnson, Timothy M. (Department: 2625)
Image analysis
Image compression or coding
Interframe coding
C375S240160
Reexamination Certificate
active
06690835
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The invention relates to video encoding methods with variable video frames.
2. Description of the Related Technology
A video information stream comprises a time sequence of video frames. The time sequence of video frames can be recorded for instance by a video camera/recorder. Each of the video frames can be considered as a still image. The video frames are represented in a digital system as an array of pixels. The pixels comprises luminance or light intensity and chrominance or color information. The information is stored in a memory of the digital system. For each pixel some bits are reserved. From a programming point of view each video frame can be considered as a two-dimensional data type, although the video frames are not necessary rectangular. Note that fields from an interlaced video time sequence can also be considered as video frames.
A particular aspect of the considered video frames is that they are variable in size and even location with respect to a fixed reference such as, e.g., the display. Moreover, the considered video frames support the object concept by indicating whether a pixel belongs to an object or not.
In principle when the video information stream must be transmitted between two digital systems, this can be realized by sending the video frames sequentially in time, for instance by sending the pixels of the video frames and thus the bits representing the pixels sequentially in time.
There exist, however, more elaborated transmission schemes enabling faster and more reliable communication between two digital systems the transmission schemes are based on encoding the video information stream in the transmitting digital system and decoding the encoded video information stream in the receiving digital system. Note that the same principles can be exploited for storage purposes.
During encoding the original video information stream is transformed into another digital representation the digital representation is then transmitted. While decoding the original video information stream is reconstructed from the digital representation.
The MPEG-4 standard defines such a transmission (and storage) efficient encoded digital representation of a video information stream.
Encoding requires operations on the video information stream, the operations are performed on a digital system (for instance in the transmitting digital system). Each operation performed by a digital system consumes power. The way in which the operations for encoding are performed is called a method. The methods have some characteristics such as encoding speed and the overall power consumption needed for encoding.
The digital system can either be application-specific hardware or a programmable processor architecture. It is well-known that most power consumption in the digital systems, while performing real-time multi-dimensional signal processing such as video stream encoding on the digital systems, is due to the memory units in the digital systems and the communication path between the memory units. More precisely individual read and write operations from and to memory units by processors and/or datapaths and between memories become more power expensive when the memory units are larger, and so does the access time or latency from the busses. Naturally also the amount of read and write operations are determining the overall power consumption and the bus loading. The larger the communication path the larger is also the power consumption for a data transfer operation. With communication is meant here the communication between memory units and the processors and data paths found in the digital system and between memories themselves. There is also a difference between on- and off-chip memories. Note that the same considerations are valid when considering speed as a performance criterion.
As the power consumption of the digital system is dominated by read and write operations, thus manipulations on data types, such as video frames, the methods are considered to be data-dominated.
As the algorithm specification, the algorithm choice and its implementation determine the amount of operations and the required memory sizes it is clear that these have a big impact on the overall power consumption and other performance criteria such as speed and bus loading.
A method for encoding a video information stream, resulting in a minimal power consumption of the digital system on which the method is implemented, and exhibiting excellent performance, e.g., being fast, must be based on optimized data storage, related to memory sizes, and data transfer, related to the amount of read and write operations. Such a method can be developed by transforming an initial less power optimal method by using various code manipulations. Such a transformation approach must be supported by an adequate exploration methodology.
In general a method can be described as an ordered set of operations which are repetitively executed. The repetition is organized in a loop. During execution data is consumed and produced. The code manipulations can be loop- and/or data-flow transformations. The transformations change the ordering of the operations in the loop and result in another data consumption-production ordering. Also data reuse concepts can be used in order to obtain a more power consumption and speed optimal method. Data reuse deals with specifying from and to which memory data is read and written. More in particular applying the data reuse concept means making copies of data to smaller memories and to let the data be accessed by the processors and/or datapaths from the smaller memories.
Naturally when such a power consumption and speed optimal encoding method exist it can be implemented on a digital system, adapted for the method. This adaptation can be done by an efficient programming of programmable (application specific) processor architectures or by actually designing an application-specific or domain-specific processor with the appropriate memory units.
The fact that the power consumption is heavily dominated by data storage and data transfer of multi-dimensional data types is demonstrated in the publication [F. Catthoor, F. Franssen, S. Wuytack, L. Nachtergaele, H. De Man, “Global communication and memory optimizing transformations for low power signal processing systems”, IEEE workshop on VLSI signal processing, La Jolla Calif., October 1994] and [R. Gonzales, M. Horowitz, “Energy dissipation in general-purpose microprocessors”, IEEE J. Solid-state Circ., Vol.SC-31, No.9, pp.1277-1283, September 1996] for custom hardware and programmable processors respectively.
Power consumption in deep submicron CMOS digital devices is dominated by the charging of wires on-chip and off-chip. The technological evolution aims at minimizing the power consumption by lowering the supply voltages, using short thin wires and small devices, using reduced logic swing. These non-application specific approaches do not exploit the characteristics of the application in the design of the digital system and/or implementation on a given digital system.
Some following general principles for power consumption reduction are known: match architecture and computation, preserve locality and regularity inherent in the application, exploit signal statistics and data correlations and deliver energy and performance on demand. These guidelines must however be translated and extended for a more memory related context as found in multi-media applications.
The data storage and transfer exploration methodology, applied for constructing the encoding methods presented in the invention, is discussed in the detailed description of the invention.
The different aspects of the invention will be illustrated for encoding following the MPEG-4 standard, discussed in the detailed description of the invention. The current realizations of MPEG based video coding multi-media applications can be distinguished in two main classes: the customized architectures and the programmable architectures.
The disadvantages of the cust
Brockmeyer Erik
Catthoor Francky
Interuniversitair Micro-Elektronica Centrum (IMEC vzw)
Johnson Timothy M.
Knobbe Martens Olson & Bear LLP
LandOfFree
System and method of encoding video frames does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method of encoding video frames, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method of encoding video frames will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3294032