Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
2000-01-06
2002-02-19
Kelley, Chris (Department: 2613)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
C348S144000, C348S145000
Reexamination Certificate
active
06349114
ABSTRACT:
BACKGROUND OF THE INVENTION
The invention relates to a camera motion parameters estimation method, said parameters being intended to become descriptors in the MPEG-7 video indexing framework.
The invention relates to a camera motion parameters estimation method, said parameters being intended to become descriptors in the MPEG-7 video indexing framework.
The last decades have seen the development of large databases of information, accessible to many people. These databases are composed of several types of media such as text, images, sound, etc . . . The characterization, representation, indexing, storage, transmission and retrieval of such information constitute important issues in the usefulness of this technology. Whatever the level of sub-division at which video indexing can be contemplated, each information sub-division can be then indexed according to several criteria such as semantic information content, scene attributes, camera motion parameters, and so on. MPEG-7, also named “Multimedia Content Description Interface” and focussing on content-based retrieval problems, will standardize generic ways to describe such multimedia content, using descriptors and description schemes that would be associated to multimedia material, in order to allow fast and efficient retrieval based on various types of features, such as text, color, texture, motion and semantic content. This standard will address applications that can be either stored (on-line or off-line) or streamed (e.g. broadcast or video in the internet) and can operate in both real time and non-real time environments.
A schematic block diagram of a possible MPEG-7 processing chain, shown in FIG.
1
and provided for processing any multimedia content, includes at the coding side a feature extraction sub-assembly
11
operating on said content, a normative sub-assembly
12
, including a module
121
for yielding the MPEG-7 definition language and a module
122
for defining the MPEG-7 descriptors and description schemes, a standard description sub-assembly
13
, and a coding sub-assembly
14
. The scope of the MPEG-7 standard is the sub-assembly
12
, and the invention is located in the sub-assemblies
12
and
13
.
FIG. 1
also shows the decoding side, including a decoding sub-assembly
16
Oust after a transmission of the coded data, or a reading operation of these stored coded data), and a search engine
17
working in reply to the actions controlled by the user.
In the MPEG-7 framework, efficient tools must be developed for many subjects like scene analysis or motion analysis, and particularly methods for camera motion feature extraction. For a motion representation, two solutions can be proposed as a possible basis for global motion descriptors extraction: the perspective model, and the block matching method. The former is well suited for camera global motion, but cannot represent tridimensional translations, that have to be distinctly described each time it is possible.
Block matching motion compensation is used as a part of the predictive coding process that is widely used in video transmission for reducing the amount of information needed to encode a video sequence. Indeed, only a little fraction of an image changes from a frame to the following one, allowing a straightforward prediction from said previous frame. More precisely, each frame (i+1) is divided into a fixed number of blocks (usually square). For each block (typically of 8×8 pixels), a search is made for the most similar block in a previous reference frame (i), over a predetermined area. The search criterion is generally the search of the best matching block, giving the least prediction error, usually computed as the mean absolute difference (which is easier to compute than for instance the mean square difference). For each block (in the present example, of 8×8 pixels) located in (x,y), the predicted image (i+1) is then computed from image (i) according to the relation (1):
B(i+1)[x,y]=B(i)[x−dx, y−dy] (1)
with (dx, dy)={right arrow over (v)}=motion vector leading from B(i), in the image (i), to B(i+1), in the image (i+1).
When starting from block matching motion vectors in order to estimate camera movements, the main problem is then that the efficiency of the estimator of these vectors is only measured in terms of a coding criterion. Motion vectors do not necessarily correspond to the real motion of the scene. For example, in an area of homogeneous texture in the scene, the estimator could choose any of the blocks inside the texture, even if the motion vector is not representative of the global motion. However, although block matching represents a motion that is not always consistent, this method will be preferred, because translations have to be distinctly described each time it is possible and the perspective model is not able to do it. Starting from motion vectors thus determined, some camera parameters will then be defined. Before describing the corresponding definition method, the camera model used in the present description is first presented.
A monocular camera moving through a static environment is considered. As can be seen in
FIG. 2
, let 0 be the optical centre of the camera and OXYZ an external coordinates system that is fixed with respect to the camera, OZ being the optical axis. Let T
x
, T
y
, T
z
be the translational velocity of OXYZ relative to the scene and R
x
, R
y
, R
z
its angular velocity. If (X,Y,Z) are the instantaneous coordinates of a point P in the tridimensional scene, the velocity components of P will be:
{overscore (X)}=−T
x
−R
y
.Z+R
z
.Y (2)
{overscore (Y)}=−T
y
−R
z
.X+R
x
.Z (3)
{overscore (Z)}=−T
z
−R
x
.Y+R
y
.X (4)
The image position of P, namely p, is given in the image plane by the relation (5):
(x,y)=internal coordinates=(f X/Z, f Y/Z) (5)
(where f is the focal length of the camera), and will move across the image plane with an induced velocity:
(u
x
, u
y
)=({overscore (x)},{overscore (y)}) (6)
After some computations and substitutions, the following relations are obtained:
u
x
=
f
·
X
_
Z
-
f
·
X
·
Z
_
Z
2
(
7
)
u
x
=
f
Z
(
-
T
x
-
R
y
·
Z
+
R
z
·
Y
)
-
f
·
X
Z
2
(
-
T
z
-
R
x
·
Y
+
R
y
·
X
)
(
8
)
and
:
u
y
=
f
·
Y
_
Z
-
f
·
Y
·
Z
_
Z
2
(
9
)
u
y
=
f
Z
(
-
T
y
-
R
z
·
X
+
R
x
·
Z
)
-
f
·
Y
Z
2
(
-
T
z
-
R
x
·
Y
+
R
y
·
X
)
(
10
)
which can also be written:
u
x
(
x
,
y
)
=
-
f
Z
·
(
T
x
-
x
·
T
z
)
+
x
·
y
f
·
R
x
-
f
(
1
+
x
2
f
2
)
R
y
+
y
·
R
z
(
11
)
u
y
(
x
,
y
)
=
-
f
Z
·
(
T
y
-
y
·
T
z
)
-
x
·
y
f
·
R
y
+
f
(
1
+
y
2
f
2
)
R
x
+
x
·
R
z
(
12
)
Moreover, in order to include the zoom in the camera model, it is assumed that a zoom can be approximated by a single magnification in the angular domain. Such an hypothesis is valid if the distance of the nearest object in the scene is large compared to the change of focal length used to produce the zoom, which is usually the case.
A pure zoom is considered in FIG.
3
. Given a point located in the image plane, on (x,y) at a time t and on (x′, y′) at the next time t′, the image velocity u
x
=x′-x along x induced by the zoom can be expressed as a function of R
zoom
(R
zoom
being defined by the relation (&thgr;′−&thgr;)/&thgr;, as indicated in FIG.
3
), as shown below.
One has indeed: tan (&thgr;′)=x′/f and tan (&thgr;)=x/f, which leads to:
u
x
=x′−x=[tan(&thgr;′)−tan (&thgr;)].f (13)
The expression of tan (&thgr;′) can be written:
tan
(
θ
′
)
=
ta
Gross Russell
Kelley Chris
U.S. Philips Corporation
Vo Tung T.
LandOfFree
Camera motion parameters estimation method does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Camera motion parameters estimation method, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Camera motion parameters estimation method will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2976817