Pulse or digital communications – Bandwidth reduction or expansion
Reexamination Certificate
2000-01-21
2004-03-23
An, Shawn S. (Department: 2613)
Pulse or digital communications
Bandwidth reduction or expansion
C375S240120, C375S240160, C375S240150, C375S240240, C348S699000, C382S236000, C382S232000, C382S238000
Reexamination Certificate
active
06711209
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to video compression. In particular, the invention relates to coding of an estimated motion field and to generating motion information in a video sequence.
BACKGROUND OF THE INVENTION
Motion compensated prediction is a key element of the majority of video coding schemes. To describe the operation of motion compensated prediction it should be appreciated that each digital image contains certain set of pixels corresponding to certain parts of the image. Each pixel may be represented, for example, as intensities of Red, Green and Blue (RGB color system) or as intensities of the luminance and two chrominance components.
FIG. 1
shows illustratively two segments of an image, S
k
and S
i
, each showing a set of pixels
10
to
15
at old locations, that is in a previous image of a sequence of images. The new locations of these pixels in a current image are shown as positions
10
′ to
15
′. The change of their location, that is their motion, defines respective motion vectors v
1
k
to v
3
k
and v
1
i
v
3
i
of the pixels in these segments. At the simplest, the segments are squares or rectangles. Attentively, and in legacy schemes, they may also be of an arbitrary form, as shown in FIG.
1
.
FIG. 2
is a schematic diagram of an encoder for compression of video sequences using motion compensation. Essential elements in the encoder are a motion compensated prediction block
1
, a motion field estimation block
2
and a motion field coder
3
. The operating principle of motion compensating video coders is to compress the prediction error E
n
(x, y), which is a difference between the incoming frame I
n
(x, y) being coded, called the current frame, and a prediction frame Î
n
(x, y), wherein:
E
n
(
x, y
)=
I
n
(
x, y
)−Î
n
(
x, y
) (1)
The prediction frame Î
n
(x, y) is constructed by the motion compensated prediction block
1
and is built using pixel values of the previous, or some other already coded frame denoted Ĩ
ref
(x, y), called a reference frame, and the motion vectors of pixels between the current frame and the reference frame. Motion vectors are calculated by the motion field estimation block
2
and the resulting vector field is then coded in some way before being provided as an input to the prediction block
1
. The prediction frame is then:
Î
n
(
x, y
)=
Ĩ
ref
[x+{tilde over (&Dgr;)}x
(
x, y
),
y+{tilde over (&Dgr;)}y
(
x, y
)] (2)
{tilde over (&Dgr;)}x(x, y) and {tilde over (&Dgr;)}y(x, y) are the values of horizontal and vertical displacement of pixel in location (x, y) and the pair of numbers [{tilde over (&Dgr;)}x(x, y), {tilde over (&Dgr;)}y(x, y)] is called the motion vector of that pixel. The set of motion vectors of all pixels in the current frame I
n
(x, y) is called a motion vector field. The coded motion vector field is transmitted as motion information to the decoder together with encoded prediction error information.
In the decoder, shown in
FIG. 3
, the current output frame Ĩ
n
(x, y) is reconstructed by finding the pixels' prediction Î
n
(x, y) in the reference frame Ĩ
ref
(x, y) and adding a decoded prediction error Ê
n
(x, y). The motion compensated prediction block
21
generates the prediction frame using the received motion information and the reference frame Ĩ
ref
(x, y). The prediction error decoder
22
generates the decoded prediction error Ê
n
(x, y) for adding to the prediction frame, the result being the current output frame Ĩ
n
(x, y).
The general object of motion compensated (MC) prediction is to minimize amount of information which needs to be transmitted to the decoder together with the amount of prediction error measured, e.g., as the energy of E
n
(x, y).
The document H. Nguen, E. Dubois, “Representation of motion information for image coding”. Proc. Picture Coding Symposium '90, Cambridge, Mass., Mar. 26-18, 1990, pages 841-845, gives a review of motion field coding techniques. As a rule of thumb, reduction of prediction error requires a more sophisticated motion field model, that is, more bits must be used for its encoding. Therefore, the overall goal of video encoding is to encode the motion vector field as compactly as possible while keeping the measure of prediction error as low as possible.
The motion field estimation block
2
, shown in
FIG. 2
, calculates motion vectors of all the pixels of a given image segment minimizing some measure of prediction error in this segment, for example square prediction error.
Due to the very large number of pixels in the frame, it is not efficient to transmit a separate motion vector for each pixel. Instead, in most video coding schemes, the current frame is divided into larger image segments so that all motion vectors of the segment can be described by few parameters. Image segments may be square blocks, for example, 16×16 and 8×8 pixel blocks are used in codecs in accordance with international standards ISO/IEC MPEG-1, MPEG-2, MPEG-4 or ITU-T H.261 and H.263, or they may comprise arbitrarily shaped regions obtained for instance by a segmentation algorithm. In practice, segments include at least few tens of pixels.
In order to compactly represent the motion vectors of the pixels in a segment, it is desirable that the motion vectors are described by a function of few parameters. Such a function is called a motion vector field model. A known group of models is linear motion model, in which motion vectors are represented by linear combinations of motion field basis functions. In such models, the motion vectors of image segments are described by a general formula:
Δ
x
(
x
,
y
)
=
∑
i
=
1
N
c
i
f
i
(
x
,
y
)
Δ
y
(
x
,
y
)
=
∑
i
=
N
+
1
N
+
M
c
i
f
i
(
x
,
y
)
,
(
3
)
where parameters c
i
are called motion coefficients and are transmitted to the decoder. In general, the motion model for a segment is based on N+M motion coefficients. Functions f
i
(x, y) are called motion field basis functions which and are known both to the encoder and decoder. Known motion field estimation techniques vary both in terms of the model used to represent the motion field and in the algorithm for minimization of a chosen measure of the prediction error.
Both the amount and the complexity of the motion varies between frames and between segments. In one case some of the content of the image may be rotated, skewed and shifted from one side of the image to the opposite side of the image. On the other hand, in another case a video camera may turn slowly about its vertical axis so that all the pixels move slightly in horizontal plane. Therefore, it is not efficient to always use all N+M motion coefficients per segment.
One way to reduce motion information is simply to reduce the number of motion coefficients from the motion field model that models the motion of pixels' locations from one image to another. However, the prediction error tends to increases, as the motion field model becomes coarser.
For every segment, it is necessary to determine the minimum number of motion coefficients that yields a satisfactorily low prediction error. The process of such adaptive selection of motion coefficients is called motion coefficient removal. This process is performed in the encoder by the motion field coding block
3
, see FIG.
2
. It is performed after motion field estimation performed by the motion field estimation block
2
.
In future, digital video transmission will be possible between wireless mobile terminals. Usually such terminals have limited space for additional components and operate by a battery so that they are likely not accommodate computing capacity comparable to fixed devices such as desktop computers. Therefore, it is crucial that the motion field coding performed in a video coder is computationall
Karczewicz Marta
Lainema Jani
An Shawn S.
Nokia Mobile Phones Ltd.
LandOfFree
Adaptive motion vector field coding does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Adaptive motion vector field coding, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Adaptive motion vector field coding will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3237472