Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
1998-03-10
2001-03-20
Le, Vu (Department: 2713)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
C348S393100, C348S717000, C375S240240
Reexamination Certificate
active
06205181
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a format for storing the color space data for video processing. More particularly, the present invention relates to a method for interleaving the color space data and a method for storing the color space data in a strip format.
2. The Background Art
With the merging of personal computer systems and entertainment systems, digital component video and audio were developed. Typically, this audio and video data are encoded into a compressed program stream for transmission. A stream demultiplexer parses the incoming program stream into audio and video bitstreams. When frames are ultimately displayed, there must be a decompression of these compressed video bitstreams. A video decoder is used for this decompression, or decoding, process.
A video stream is made up of a number of still frames, or pictures. Referring first to
FIG. 1
, a representation of a frame
10
is shown. Each frame
10
comprises a plurality of horizontal slices
12
, each of which includes a plurality of macroblocks
14
. A typical macroblock size is 16 lines×16 pixels. A frame, or picture, resolution of 720×576 is defined by 720×576 pixels which correspond to 45×36 macroblocks.
Many international standards, such as the MPEG 2 standard, are used for digital video compression and decompression. MPEG standards take advantage of the redundancy on a frame-to-frame basis of a motion video sequence. According to the MPEG 2 standard, there are three types of coded frames: I (intra) frame, P (predicted) and B(bidirectionally predicted). Frame prediction may occur using forward or backward prediction. Forward prediction occurs when a future frame is constructed from a previous frame in a sequence of frames. Backward prediction occurs when a future frame in a sequence of frames is used to predict a current or past frame in the sequence of frames. An I frame contains a stand-alone still image, and each macroblock within the I frame can be decoded without using information from another frame. As shown in
FIG. 2A
, a P frame
16
is coded relative to the nearest previous I or P frame
18
. Similarly, as shown in
FIG. 2B
, a B frame
20
is coded relative to two reference frames, the closest previous I or P frame
22
and the closest future P frame
24
.
During decoding, the first frame received is an I frame, which serves as a reference frame for later B and P frames. From the encoded video bitstream, the video decoder reconstructs the I frames. Using the I frames, the P and B frames are generated using additional encoded data and motion vectors. Therefore, P and B frames are constructed using differential data relative to this I frame.
Reconstruction of each B and P macroblock is based on motion vectors, and is performed through the addition of a differential macroblock to a reference macroblock. For example, for a B frame inter-macroblock having two references, the two reference macroblocks are averaged to produce the final macroblock. This macroblock is then added to the differential macroblock.
Each macroblock comprises a plurality of pixels, each of which is defined by color space components. A color space is a mathematical representation for a color. Different color spaces which provide different ways of representing a color which will ultimately be displayed in a video system. For example, the red, green, and blue (RGB) color space is commonly used in computer graphics. Similarly, the YUV color space represents the luma component Y, or black and white portion, as well as the color difference components U and V. A macroblock in YUV format contains data for all Y, U, V components. Since many video compression standards compress data on a macroblock by macroblock basis, the compressed data stream must also be decoded on a macroblock by macroblock basis. In this manner, the encoding process is reversed to display the encoded information.
Digital encoded video data must be stored and processed during video decoding. Data storage systems used for internal storage during video decoding can optimize data accesses during video decoding. In addition, various data access modes may be used to increase the efficiency of the decoding process.
Sequential transfer modes, such as a burst mode, are often used to improve system throughput in data intensive applications. In this manner, the speed of movement of large sequential blocks of data is substantially increased. When a burst access mode is used, addresses are incremented to read each sequentially stored byte of data. However, a jump to a wholly different location in memory cannot be performed to read the next byte. The full capability of a sequential transfer mode, therefore, can be realized only through the sequential storage of each byte of data.
Pixels in each macroblock are traditionally stored in blocks since they are compressed. Three types of macroblocks are available in MPEG 2 . The 4:2:0 macroblock consists of four Y blocks, one U block, and one V block. In the 4:2:0 chroma format, for each 16×16 pixel Y block, the corresponding U and V blocks have size 8×8 pixels. In other words, for every 4 Y pixels, 1 U and 1 V pixel are shared. Most MPEG 2 decoders use the 4:2:0 chroma format for internal storage.
A 4:2:2 macroblock consists of four Y blocks, two U blocks, and two V blocks. In the 4:2:2 format, each 16×16 pixel Y block is associated with one U and one V block having size 16×8 pixels. In this format, 2 Y pixels share 1 U and 1 V pixel.
A 4:4:4 macroblock consists of four Y blocks, four U blocks, and four V blocks. Each 16×16 pixel Y block is associated with one U and one V block of size 16×16. Therefore, the 4:4:4 format stores an equal number of Y, U and V pixels.
Data formats typically used to store color space components during the MPEG 2 decoding process prevent large sequential blocks from being transferred during a sequential transfer mode. Planar data formats are typically used to store the YUV data in memory during the decoding process. In planar formats, each of the color space components is stored in horizontal scan line order in memory, and these blocks of memory may or may not be adjacent. Since all of the bytes within each color space component are not stored sequentially in the planar formats, only short burst reads or writes can be performed to read each line. Therefore, the advantages of a sequential transfer mode cannot be fully realized.
For example, in the 4:2:0 chroma format, for each 16×16 pixel Y block, the corresponding U and V blocks have size 8×8 pixels. Assuming each pixel comprises 1 byte, it takes 16 burst accesses to read or write a 16×16 pixel Y block, with each burst access reading or writing 16 bytes. In addition, it takes another 2×8=16 burst accesses to read or write the U and V blocks. Thus, a total 32 burst accesses are needed for each macroblock access.
Using the 4:2:2 chroma format, each 16×16 pixel Y block is associated with one U and one V block having size 16×8 pixels. Therefore, 16 burst accesses are necessary for the Y block, the U block and the V block, totaling 48 burst accesses for each macroblock.
In the 4:4:4 chroma format, each 16×16 pixel Y block is associated with one U and one V block of size 16×16. Since the number of lines in each U or V block is identical to that of the 4:2:2 format, 48 burst accesses are still required for each macroblock. The only difference between the 4:2:2 and 4:4:4 formats is that each U or V access reads or writes 16 bytes in the 4:4:4 format instead of 8 bytes in the 4:2:2 format.
In a software based decoder, the probability that required data is in a cache is called a “hit rate”. If the data is not found in the cache, a cache “miss” occurs. If there is a cache “miss”, the processor must then access the main memory. The probability of a “hit” is complex, and is related to the size of the block of data, as well as other factors.
As described above, when the data is stored in the planar form
Hu Xiaoping
Kang David Sokmin
Nguyen Hungviet H.
Chips & Technologies, LLC
Le Vu
Ritchie David B.
Thelen Reid & Priest LLP
LandOfFree
Interleaved strip data storage system for video processing does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Interleaved strip data storage system for video processing, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Interleaved strip data storage system for video processing will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2454627