Pulse or digital communications – Bandwidth reduction or expansion – Television or motion video signal
Reexamination Certificate
2000-03-27
2002-04-09
Le, Vu (Department: 2613)
Pulse or digital communications
Bandwidth reduction or expansion
Television or motion video signal
Reexamination Certificate
active
06370196
ABSTRACT:
RELATED APPLICATION
The application is related to the following applications assigned to the same applicant as the present invention and filed on even date herewith, the disclosures of which are hereby incorporated by reference:
Method and apparatus for compressing video sequences (Our file: IDT 018 WO). Method and apparatus for compression of video images and image residuals (Our file: IDT 018 WO).
FIELD OF INVENTION
This patent deals with the field of motion estimation in sequences of two-dimensional images with arbitrary shapes over several frames where no restriction on the type of image data is given. Image sequences can be acquired for instance by video, X-ray, infrared, radar cameras or by synthetic generation etc.
BACKGROUND OF INVENTION
Motion estimation is a highly under-determined problem, therefore additional constraints are necessary in order to get a unique solution for the corresponding system of equations. In many approaches isotropic or anisotropic spatial smoothing terms are used for this purpose. But this is still not sufficient to get satisfying results for real sequences. For tracking motion over several frames, detecting motion vectors with high amplitudes, overcoming the “aperture problem” and aliasing effects in time, stabilizing the motion estimation against outliers and noise and getting high correlated motion estimates in time and space enhanced prediction and filtering methods have to be applied. Although a lot of work has been done in the framework of estimating dense motion fields, a conclusive, detailed treatment of arbitrary shaped images is hardly described, especially for hierarchical motion estimation systems. For general reference see the following reference list:
1. Joachim Dengler. Local motion estimation with the dynamic pyramid. Pyramidal Systems for Computer Vision, F25:289-297, 1986. Comment: Presentation of a pyramidal approach.
2. Enkelmann. Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences. Computer Vision, Graphics and Image Processing, 43:150-177, March 1988. Comment: Applying multigrid methods for solving estimating optical flow fields by using orientated smoothness constraints.
3. Sugata Ghosal and Petr Vanok. A fast scalable algorithm for discontinuous optical flow estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(2), February 1996, Comment: Multigrid approach for solving the motion estimation problem by using anisotropic smoothness constraints.
4. Gonzalez and R. E. Wood. Digital Image Processing. Addison Wesley, 1992. Comment: General image processing book.
5. Sheila S. Hemami Gregory U. Conklin. Multi-resolution motion estimation. In IEEE ICASSP München, pages 2873-2876, 1997. Comment: Coarse to fine propagation versus fine to coarse propagation.
6. B. K. P Horn and B. G. Schunck. Determining optical flow. Artificial Intelligence, 17:185-203, 1981. Comment: Basic article for gradient based approaches.
7. Bernd Jaehne. Digitale Bildverarbeitung. Springer-Verlag, 1993. Comment: General book about image processing. General description of pyramidal approaches.
8. P. Anandan; J. R. Bergen and K. J. Hanna. Hierarchial model-based motion estimation. In Reginald L. Lagendijk M. Ibrahim Sezan, editor, Motion Analysis and Image Sequence Processing. Kluwer Academic Publishers, 1993. Comment: Introduction to the advantage of using pyramidal approaches for determining optical flow.
9. Hans-Helmut Nagel. Image sequences—ten (octal) years—from phenomenology towards a theoretical foundation. IEEE, pages 1174-1185, 1986. Comment: Overview article.
10. P. Anandan. A unified perspective on computational techniques for the measurement of visual motion. IEEE, Conference on Computer Vision, pages 219-230, 1987. Comment: Overview of the problems and possibilities of pyramidal approaches for motion estimation.
11 . Adelson P. J. Burt. The laplacian pyramid as a compact image code. IEEE Trans. Communications, 31:532-540, 1983. Comment: Introduction to pyramids.
12. Singh. Optic Flow Computation, A Unified Perspective. IEEE Computer Society Press Monograph, 1991. Comment: General introduction and presentation of a framework for motion estimation.
13. T. Lin and J. L. Barron. Image reconstruction error for optical flow. from Internet, 1996. Comment: Comparison of different motion estimators.
14. Woods and J. Kim. Motion compensated spatial temporal kalman filter. In Reginald L. Lagendijk M. Ibrahim Sezan, editor, Motion Analysis and Image Sequence Processing. Kluwer Academic Publishers, 1993. Comment: Noise reduction in image sequences by using the time correlation between images. The method is a combination of motion compensation and spatial temporal Kalman filtering.
15. B. Chupeau, M. Pecot. Method for hierarchical estimation of the movement in asequence of images, U.S. Pat. No. 5,278,915, issued Jan. 11, 1994, Thomson-CSF, Puteaux, France.
16. V. Markandey. System and method for determining optical flow, U.S. Pat. No. 5,680,487, issued Oct. 21, 1997, Texas Instruments Incorporated, Dallas, Tex.
Objects of Invention
It is an object of this invention to provide mechanisms for improving motion estimation between arbitrary shaped images where large displacement amplitudes may occur. The improvements concern for example the quality of images predicted from the motion fields (i.e. a reduction of the displaced frame differences) and the temporal and spatial correlation of the motion fields performing motion estimation within a set of subsequent images. The improvement of temporal and spatial correlation can be useful in image analysis and compression of motion fields.
It is an object of the invention to provide hierarchical systems which are able to estimate dense motion fields between arbitrary shaped images. The explicit treatment of the shapes as described in the present invention allows a natural consideration of invalid pixels which may occur during the estimation process.
It is an object of the invention to provide methods which are applicable in motion estimation schemes where an image is predicted by forward warping as well as for motion estimation schemes where an image is predicted by backward warping.
It is a further object of the present invention to provide a technique for motion estimation in a sequence of related images. The images can be related in any way, for instance temporal or spatial (i.e. in subsequent resolutions).
It is a further object of this invention to provide tracking of motion for several frames where large displacement amplitudes may occur.
It is a further object of this invention to provide a technique for combining motion fields achieved by different estimations.
It is a further object of this invention to provide a technique for propagating information in a subsequent estimation process.
It is a further object of this invention to provide a technique for a local adaptive filtering of motion fields in order to achieve a gain in quality.
It is a further object of this invention to provide a technique for using motion fields from former estimations as hypotheses for the following estimation.
Notations and Definitions
D
v
: Vertical component of the motion field.
D
h
: Horizontal component of the motion field.
D: All components of the displacement field, i.e. the motion field.
D:=(D
v
, D
h
) for two dimensions.
H
v
: Vertical component of a hypothesis for the motion field.
H
h
: Horizontal component of a hypothesis for the motion field.
H: All components of the hypothesis for the motion field.
H:=(H
v
, H
h
) for two dimensions.
I
D
: Image in the coordinate system of the motion field D.
S
D
: Shape field in the coordinate system of the motion field D. It is a validity field which defines the valid pixels for all fields in the position (coordinate system) of D.
I
T
: Image in target position, i.e. the image “to” which the motion field points.
S
T
: Shape field in target position. It is a validity field which defines the valid pixels for all fields in the target position.
{circumflex over (X)}: A field X which is created by forwar
Griessl Max
Martens Harald Skt.
Reberg Jan Otto
Röttgermann Clemens
Wittkop Markus
IDT International Digital Technologies Deutschland GmbH
Le Vu
LandOfFree
Method and apparatus for multiresolution object-oriented... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for multiresolution object-oriented..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for multiresolution object-oriented... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2922252