Image analysis – Image transformation or preprocessing
Reexamination Certificate
1999-08-04
2002-12-17
Couso, Jose L. (Department: 2621)
Image analysis
Image transformation or preprocessing
C382S284000, C345S683000
Reexamination Certificate
active
06496606
ABSTRACT:
FIELD OF THE INVENTION
The invention relates to a method for generating a large static image M(n), such as a sprite or a mosaic, from a video sequence including successive video objects VOs, said method comprising the steps of:
(A) estimating motion parameters related to the current video object V
0
(n) of the sequence, with respect to the previously generated static image M(n−1);
(B) warping said current video object V
0
(n), on the basis of said estimated motion parameters;
(C) blending the warped video object WV
0
(n) thus obtained with the previously generated static image M(n−1); and to a corresponding device. This invention may be useful in relation with the MPEG-4 and MPEG-7 standards.
BACKGROUND OF THE INVENTION
The MPEG-7 standard has for object to standardize within some years generic ways to describe multimedia content, in view of a fast and efficient retrieval of data on the basis of various types of features such as text, color, texture, motion and semantic content. In this context, a mosaic can play a useful role. As indeed explained for instance in the article “Efficient representations of video sequences and their applications”, M. Irani and al., Signal Processing: Image Communication, vol.8, 1996, pp.327-351, a mosaic image is constructed from all frames in a scene sequence, giving a panoramic view of said scene. It is possible to extract for instance from this panoramic view the main features of the sequence, such as chrominance or luminance histograms, objects shapes, global motion parameters, and so on.
The definition of a mosaic may be compared to that of a sprite, as used in the context of the MPEG-4 standard. As described for instance in the case of a sprite in the document WO 98/59497, which can be also applied to the case of a mosaic, three main steps may compose a sprite or mosaic generation (in the following, the generic word “static image” will therefore be used in place of sprite or mosaic). A motion estimation step is first provided, in order to find the motion parameters that allow to merge correctly a current frame F(n) with the static image M(n−1) already composed of the previous frames F(
1
), F(
2
), . . . , F(n−1). The inverse parameters are then computed, so that the current frame may be compensated in their direction; this second step is also called warping. The warped current frame F(n) is finally blended with M(n−1) in order to form a new accreted static image M(n), with which the next incoming frame F(n+1) will be merged, and so on.
SUMMARY OF THE INVENTION
It is an object of the invention to propose a static image generation method with an increased robustness to outliers.
To this end, the invention relates to a method such as described in the introductory paragraph of the description and which is moreover characterized in that:
(1) said method also comprises, between said warping and blending steps, an additional step for computing, for each picture element of the current video object V
0
(n), a weighting coefficient w
WF(n)
[x,y] correlated to the error between the warped video object WV
0
(n) and the static image M(n−1) at each picture element [x,y];
(2) the blending step provided for determining the newly generated static image M(n) takes into account said weighting coefficients according to the following blending formula, used for each pixel [x,y]:
M
(
n
)
[
x
,
y
]
=
w
M
(
n
-
1
)
[
x
,
y
]
·
M
(
n
-
1
)
[
x
,
y
]
+
w
WF
(
n
)
[
x
,
y
]
·
WF
(
n
)
[
x
,
y
]
w
M
(
n
-
1
)
+
w
WF
(
n
)
where the definitions of the terms are the following:
(a) n>0
(b) whatever (x,y), w
M(0)
=0;
(c) whatever (x,y), w
WF(n)
[x,y]=
w
WF
(
n
)
[
x
,
y
]
=
1
r
(
x
,
y
)
ⅆ
ⅆ
r
ρ
(
r
(
x
,
y
)
)
;
(d) w
M(n)
=w
M(n−1)
+w
WF(n)
.
It is another object of the invention to propose a static image generation device for carrying out said method.
To this end, the invention relates to a device for generating a large static image M(n), such as a sprite or a mosaic, from a video sequence including successive video objects VOs, said device comprising:
(A) a motion estimation circuit, provided for estimating a motion information related to the relative motion between the current video object V
0
(n) of the sequence and the previously generated static image M(n−1);
(B) a warping circuit, provided for defining on the basis of said current video object and said motion information a warped video object WVO(n);
(C) a blending circuit, provided for updating the static image, said previously generated static image M(n−1) being replaced by the new one M(n) generated by blending said previously generated static image M(n−1) and said warped video object; characterized in that:
(1) said device also comprises a pixel-based weighting circuit, receiving the current video object V
0
(n) and the associated motion information and computing for each picture element of said current video object a weighting coefficient that depends on the error between the warped video object WVO(n) and the previously generated static image M(n−1) at each picture element [x,y];
(2) the blending formula used in the blending circuit for determining the newly generated static image M(n) takes into account the computed weighting coefficients according to the following blending formula, used for each pixel [x,y]:
M
(
n
)
[
x
,
y
]
=
w
M
(
n
-
1
)
[
x
,
y
]
·
M
(
n
-
1
)
[
x
,
y
]
+
w
WF
(
n
)
[
x
,
y
]
·
WF
(
n
)
[
x
,
y
]
w
M
(
n
-
1
)
+
w
WF
(
n
)
where the definitions of the terms are the following:
(a) n>0
(b) whatever (x,y), w
M(0)
=0;
(c) whatever (x,y),
W
WF
(
n
)
[
x
,
y
]
=
1
r
(
x
,
y
)
ⅆ
ⅆ
r
ρ
(
r
(
x
,
y
)
)
;
(d) w
M(n)
=w
M(n−1)
+w
WF(n)
.
The advantageous principle of the proposed method and device is that it is possible to accumulate confidence in the weight static image, in order not to take into account picture elements with low weight (i.e. outliers), which is carried out by summing for each picture element in the weight static image weights of the same picture element.
REFERENCES:
patent: 5943445 (1999-08-01), Dufaux
patent: 5999662 (1999-12-01), Burt et al.
patent: 6037988 (2000-03-01), Gu et al.
Bonnet Michel
Boutroux Vincent
Couso Jose L.
Gross Russell
Koninklijke Philips Electronics , N.V.
Patel Kanji
LandOfFree
Static image generation method and device does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Static image generation method and device, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Static image generation method and device will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2978112