Image analysis – Image transformation or preprocessing
Reexamination Certificate
1999-08-04
2002-11-26
Boudreau, Leo (Department: 2621)
Image analysis
Image transformation or preprocessing
C382S284000, C345S683000
Reexamination Certificate
active
06487323
ABSTRACT:
FIELD OF THE INVENTION
The invention relates to a method for generating a large static image, such as a sprite or a mosaic, from a video sequence including successive video objects VOs, said method comprising, in view of the accretion of said static image, the steps of:
(A) estimating motion parameters related to the current video object V
0
(n) of the sequence, with respect to the previously generated static image;
(B) warping said current video object V
0
(n), on the basis of said estimated motion parameters;
(C) blending the warped video object WV
0
(n) thus obtained with said previously generated static image; and to a corresponding device. This invention may be useful in relation with the MPEG-4 and MPEG-7 standards.
BACKGROUND OF THE INVENTION
The descriptors and description schemes that will be standardized within the frame of MPEG-7 (MPEG-7 has for object to standardize within some years generic ways to describe multimedia content) will allow fast and efficient retrieval of data, on the basis of various types of features such as text, color, texture, motion and semantic content. In this MPEG-7 context, a mosaic can also play a useful role, as it will be shown.
Sequences, video shots and key-frames follow a hierarchical structure: a video shot is a particular sequence which shows a single background, while a key-frame is a visual representation in only one image of this shot. A visual representation of a video sequence can be obtained by the extraction of key-frames from a previous shot partition of the whole sequence. The process then chooses as key-frame one image of each shot, so that it only shows 25 a part of the video shot that may not be the most reliable one for representation. A mosaic seems however to be a better choice than a key-frame, when it is wanted to show the whole video shot in a single panoramic view of background information. As explained for instance in the article “Efficient representations of video sequences and their applications”, M. Irani and al., Signal Processing: Image Communication, vol.8, 1996, pp.327-351, a mosaic image is a kind of large static image constructed from all frames in a scene sequence, giving a panoramic view of said scene. From this panoramic view, it is then possible to extract for instance the main features of the sequence, such as chrominance or luminance histograms, objects shapes, global motion parameters, and so on (all these features constitute relevant standard descriptors for MPEG-7 and are useful for MPEG-7 compliant search engines).
The definition of a mosaic may be compared to that of a sprite, as used in the context of the MPEG-4 standard. A sprite is a large static image composed of the pixels in an object visible through an entire sequence. This static image forms a panoramic view whose some portions may not be visible in some frames because of foreground objects or camera motion. If all the relevant pixels throughout the entire sequence are collected, a complete panoramic view (called background sprite) is obtained, which can be efficiently transmitted (or stored) and used later for re-creating portions of frames.
As described for instance in the case of a sprite in the document WO 98/59497 (but this description can be applied to the case of a mosaic), three main steps may compose a sprite or mosaic generation (in the following, the generic word “static image” will be therefore used in place of sprite or mosaic). A motion estimation step is first provided, in order to find the motion parameters that will allow to merge correctly a current frame F(n) with the static image M(n−1) already composed of the previous frames F(1), F(2), . . . , F(n−1). The inverse parameters are then computed, so that the current frame may be compensated in the direction of these inverted parameters ; this second step is also called warping. The warped current frame F(n) is finally blended with M(n−1) in order to form a new accreted static image M(n), with which the next incoming frame F(n+1) will be merged, and so on.
However, an observation of the obtained static image may lead to consider that some parts of said static image are not clean. For instance, some parts of a moving object have not been completely removed.
SUMMARY OF THE INVENTION
It is therefore an object of the invention to propose a method allowing to generate a static image without such artefacts.
To this end, the invention relates to a method such as described in the introductory paragraph of the description and which is moreover characterized in that:
(1) said method also comprises (n−1) further accretion steps applied to the same video sequence considered in the reverse order, each additional accretion step itself including a warping sub-step, based on each successive video object considered in said reverse order and on the corresponding estimated motion parameters, and a blending sub-step, provided between the warped video object thus considered and the static image generated at the end of the previous accretion step.
It is another object of the invention to propose a static image generation device for carrying out said method.
To this end, this invention relates to a device for generating a large static image, such as a sprite or a mosaic, from a video sequence including successive video objects VOs, said device comprising, in view of the accretion of said static image in a first accretion stage:
(A) a motion estimation circuit, provided for estimating a motion information related to the relative motion between the current video object VO(n) of the sequence and the previously generated static image;
(B) a first warping circuit, provided for defined on the basis of said current video object and said motion information a warped video object WVO(n);
(C) a first blending circuit, provided for blending the warped video object WVO(n) thus obtained with said previously generated static image, said previously generated static image being thus updated by replacement by the new one;
characterized in that it also comprises at least a further accretion loop including a second warping circuit, a second blending circuit provided for blending the warped video object thus obtained with the previously generated static image, and a memory for storing said generated static image, said memory and said second warping and blending circuits being organized for carrying on the updated static image available at the output of the first blending circuit (n−1) additional accretion steps taking into account on the one hand the same video sequence but considered in the reverse order and on the other hand the estimated motion information corresponding to each concerned video object of said reverse sequence, and, between said warping and blending circuits of said first accretion stage or of anyone of said further accretion loops, means for computing for each picture element a weighting coefficient correlated to the error between the warped video object and the generated static image and taken into account by the blending circuit during the blending step.
REFERENCES:
patent: 5943445 (1999-08-01), Dufaux
patent: 5999662 (1999-12-01), Burt et al.
patent: 6037988 (2000-03-01), Gu et al.
patent: 6097854 (2000-08-01), Szeliski et al.
Bonnet Michel A. A.
Sans Estrada Oriol
Boudreau Leo
Gross Russell
Koninklijke Philips Electronics , N.V.
Patel Kanji
LandOfFree
Static image generation method and device does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Static image generation method and device, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Static image generation method and device will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2923450