Image analysis – Image transformation or preprocessing – Changing the image coordinates
Reexamination Certificate
1998-11-16
2001-06-19
Couso, Yon J. (Department: 2623)
Image analysis
Image transformation or preprocessing
Changing the image coordinates
C382S232000
Reexamination Certificate
active
06249617
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates generally to video coding, and in particular, to a video encoder having an integrated scaling mechanism.
2. Background Art
Generally, there are two types of compression: (1) moving picture compression (also known as motion estimation), and (2) still image compression. Some commonly used standards for moving picture compression include MPEG-I, MPEG-II, H.261, etc. The dominant standard for still image compression is the JPEG standard. Both moving picture compression and still image compression utilize discrete cosine transform (DCT) and variable length encoding (VLE) to compress data (i.e., eliminate the spatially redundant data and de-compress the data).
The moving picture compression utilizes temporal data to compress the data further. Specifically, moving picture compression employs motion estimation techniques that refer to data in previous and/or future frames, also known as B (bidirectional) and P (previous) frames, in an image. The basic scheme is to predict motion from frame to frame in the temporal direction, and then to use DCTs to organize the redundancy in the spatial directions.
In contrast, still image compression utilizes the current frame without considering a previous frame. In other words, still image compression only utilizes an I-frame (intra-frame) without referring to the B and P frames. Additional information regarding the JPEG Still Picture Compression Standard is provided in a paper entitled “The JPEG Still Picture Compression Standard”, by Gregory Wallace of Digital Equipment Corp., submitted in December 1991 for publication in IEEE Transactions on Consumer Electronics (a copy is attached herewith).
Up-scaling of an image is important in still image compression and not in moving picture compression because data related to moving picture compression in a DCT buffer is a residue of the motion estimation data and cannot be scaled up.
FIG. 1
illustrates the general structural blocks that are used for, and the steps involved in, the conventional digital coding of a sequence of video images. In particular, the video image is made up of a sequence of video frames
10
that are captured, such as by a digital camera, and transmitted to a video encoder
12
. The video encoder
12
receives the digital data on a frame-by-frame and macroblock-by-macroblock basis, and applies a video encoding algorithm to compress the video data. In some applications, the video encoding algorithm can also be implemented in hardware. The video encoder
12
generates an output which consists of a binary bit stream
14
that is processed by a modulator
16
. The modulator
16
modulates the binary bit stream
14
and provides the appropriate error protection. The modulated binary bit stream
14
is then transmitted over an appropriate transmission channel
18
, such as through a wireless connection (e.g., radio frequency), a wired connection, or via the Internet. The transmission can be done in an analog format (e.g., over phone lines or via satellite) or in a digital format (e.g., via ISDN or cable). The transmitted binary bit stream
14
is then demodulated by a demodulator
20
and provided to a video decoder
22
. The video decoder
22
takes the demodulated binary bit stream
24
and converts or decodes it into sequential video frames. These video frames are then provided to a display
26
, such as a television screen or monitor, where they can be viewed. If the transmission channel
18
utilizes an analog format, a digital-to-analog converter is provided at the modulator
16
to convert the digital video data to analog form for transmission, and an analog-to-digital converter is provided at the demodulator
20
to convert the analog signals back into digital form for decoding and display.
The video encoding can be embodied in a variety of ways. For example, the actual scene or image can be captured by a camera and provided to a chipset for video encoding. This chipset could take the form of an add-on card that is added to a personal computer (PC). As another example, the camera can include an on-board chip that performs the video encoding. This on-board chip could take the form of an add-on card that is added to a PC, or as a separate stand-alone video phone. As yet another example, the camera could be provided on a PC and the images provided directly to the processor on the PC which performs the video encoding.
Similarly, the video decoder
22
can be embodied in the form of a chip that is incorporated either into a PC or into a video box that is connected to a display unit, such as a monitor or television set.
Each digital video frame
10
is made up of x columns and y rows of pixels (also known as “pels”). In a typical frame
10
(see FIG.
2
), there could be 720 columns and 640 rows of pels. Since each pel contains 8 bits of data (for luminance data), each frame
10
could have over three million bits of data (for luminance data). If we include chrominance data, each pel has up to 24 bits of data, so that this number is even greater. This large quantity of data is unsuitable for data storage or transmission because most applications have limited storage (i.e., memory) or limited channel bandwidth. To respond to the large quantity of data that has to be stored or transmitted, techniques have been provided for compressing the data from one frame
10
or a sequence of frames
10
to provide an output that contains a minimal amount of data. This process of compressing large amounts of data from successive video frames is called video compression, and is performed in the video encoder
12
.
During conventional video encoding, the video encoder
12
will take each frame
10
and divide it into blocks. In particular, each frame
10
can be first divided into macroblocks MB, as shown in FIG.
2
. Each of these macroblocks MB can have, for example, 16 rows and 16 columns of pels. Each macroblock MB can be further divided into four blocks B, each block having 8 rows and 8 columns of pels. Once each frame
10
has been divided into blocks B, the video encoder
12
is ready to compress the data in the frame
10
.
FIG. 3
illustrates the different steps, and the possible hardware components, that are used by the conventional video encoder
12
to carry out the video compression. Since each frame
10
contains a plurality of blocks B, the following steps will process each frame
10
on a block-by-block basis.
Each block B from each frame
10
is provided to a memory
42
that is provided to store the unscaled image. A separate upscaler circuit
44
reads the unscaled image from memory
42
, scales the image, and writes the scaled image back to memory
42
. As will be described later, a DCT block
60
reads the scaled image for further processing.
FIG. 5
, which is described further hereinafter, describes more fully the interaction of memory
42
, upscaler
44
, and DCT
60
.
Each block B from each frame
10
is also provided to a QP decision engine
50
which determines a QP or quantization step size number for the block or groups of blocks. This QP number is determined by a rate control mechanism which divides a fixed bit budget of a frame among different blocks, and is used by the quantization engine
80
to carry out quantization as described below.
Each block B is now provided to a DCT engine
60
. DCT of individual blocks helps in removing the spatial redundancy by bringing down the most relevant information into the lower most coefficients in the DCT domain. DCT can be accomplished by carrying out a Fourier-like transformation of the values in each block B. DCT produces a transformed block
70
in which the zeros and lower values are placed in the top left corner
72
of the transformed block
70
, and the higher frequency values are placed in the bottom right corner
74
.
After having obtained a block
70
of DCT coefficients which contain the energy of the displaced blocks, quantization of these blocks
70
is performed by quantization engine
80
. Quantization is a uniform quantization with a
Couso Yon J.
Sun Raymond
Winbond Electronics Corp.
LandOfFree
Video encounter having an integrated scaling mechanism does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Video encounter having an integrated scaling mechanism, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Video encounter having an integrated scaling mechanism will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2544372