Television – Image signal processing circuitry specific to television – Motion vector generation
Patent
1998-04-02
2000-05-02
Le, Vu
Television
Image signal processing circuitry specific to television
Motion vector generation
348416, H04N 732
Patent
active
060578920
DESCRIPTION:
BRIEF SUMMARY
The invention relates to video signal processing and to an improved method of motion correlation that can be used as part of a system to measure movement in television pictures (references 1, 3, 6, 8, 19 & 21 in the appendix). Motion estimation, in television pictures, is important because it allows a range of signal processing techniques to be used that give improved performance (references 2,3,4,5,10,11,12,13,14,15,16,18, 19 & 20.
Television signals originated by a television camera are, conventionally, interlaced. Television signals originated on film are not, fundamentally, interlaced although they are formatted on an interlaced lattice. There is potential confusion in the use of the terms `field` and `frame` used to describe television systems. To avoid this confusion the term `picture` is used throughout and can be taken to mean either field or frame depending on the application.
One way in which motion estimation can be performed is as a two stage process (references 3, 4, 13, 14, 15, 16, 18 & 19). First a moving scene is analysed to determine what movements it contains. This first stage of the analysis would produce a list of several different motions that may be present in the scene. This list might contain, for example, the distinct motion of foreground and background objects. Each of the motion vectors, produced by the first stage of analysis, is then tested to determine whether it applies to any given pixel. The first stage of analysis is non-localised and it is the second stage of analysis that locates the spatial position of the different movements. This invention is concerned with the first stage of analysis.
One, conventional, way in which movement in image sequences can be analysed is by the use of cross-correlation. Cross-correlation is performed on two successive images in a sequence. The cross-correlation function is expected to have peaks at positions corresponding to displacements between the two images. With distinct foreground and background objects in an image the cross-correlation between successive images would be expected to give two peaks corresponding to the two different movements. Unfortunately the shapes of the peaks in the cross-correlation surface depend strongly on the (2 dimensional) spectrum of the image. Since the energy in image spectra is typically concentrated at low frequencies the peaks in cross-correlation surfaces are, correspondingly, typically rounded and indistinct. The rounded shape of typical cross correlation peaks makes determining the position of the centre of the peak very difficult. Therefore, motion analysis using cross correlation is very inaccurate.
Phase correlation has been used as an improved method of motion analysis (reference 4, 18 & 19). The phase correlation function is similar to cross correlation. The phase correlation function of two successive images in a sequence would also be expected to exhibit peaks in positions corresponding to movements in the image. Phase correlation, in contrast to cross correlation, uses normalised, or `whitened` spectra prior to correlation. This gives much sharper peaks in the phase correlation surface for most images. The sharp peaks enable the displacement between two successive images to be accurately measured.
Motion correlation provides an analysis of the motion in an image sequence based on its three dimensional, spatio-temporal spectrum. The motion correlation algorithm is defined as follows. Let the brightness (or similar function of the image) be represented by g(x,y,t); where x, y & t represent the horizontal, vertical and temporal co-ordinates of the image sequence respectively. & f are horizontal, vertical and temporal frequencies respectively and F represents the Fourier transform operation. spatial power spectrum D(m,n); * represents complex conjugate. ##EQU1## where; D(m,n)=.intg.G(m,n,f).multidot.G*(m,n,f)df 3. Re-transform the normalised spatio-temporal power spectrum, N (m,n, f), to the spatio-temporal domain; F.sup.-1 represents the inverse Fourier transform operation. correlation function,
REFERENCES:
patent: 5005078 (1991-04-01), Gillard
patent: 5535288 (1996-07-01), Chen et al.
patent: 5550935 (1996-08-01), Erdem et al.
patent: 5621467 (1997-04-01), Chien et al.
"Pre-and Post-Processing in a Video Terminal Using Motion Vectors" by L. Chiariglione, L. Corgnier, M. Guglielmo.
"Review of Techniques for Motion Estimation and Motion Compensation" by Eric Dubois & Janusz Konrad. Fourth International Colloquium on Advanced Television System Proceedings. Jun. 25-29, 1990.
"Motion-compensating filed interpolation from interlaced and non-interlaced grids" by B. Girod & R. Thoma. 2nd International Technical Symposium on Optical dn Electro-Optical Applied Science & Engineering. Dec. 1985.
"Motion Compensated Interpolation" by Elana Marcozzi & Stefano Tubaro. SPIE vol. 804 Advances in Image Processing (1987), pp. 61-67.
"Model-Based Motion Estimation and its Application to Restoration and Interpolation of Motion Pictures" by Dennis Michael Martinez. MIT Technical Report No. 530, Jun. 1987. pp. 1-160.
"Motion-Compensated Television Coding: Part I" by A.N. Netravali & J.D. Robbins. The Bell System Technical Journal, vol. 58, No. 3, Mar., 1979. pp. 631-670.
"Standards Conversion Using Motion Compensation" by Thomas Reuter. Signal Processing 16 (1989)pp 73-28.
"A Motion Compensated Standards Converter for Down Conversion of 1125/60.2:1 SMPTE-240M High Definition to 625/50/2:1 Video" by J.W. Richards, S.M. Keating, C.H. Gillard. pp. 568-580.
"Experience with a prototype motion compensated standards converter for down-conversion of 1125/60/2:1 SMPTE-240M high definition to 625/50/2:1 video" by JW Richards, SM Keating, NI Saunders, CW Walters.
"Advanced High-Definition 50 to 60-Hz Standards Conversion" by P Robert, M. Lamnabhi, JJ Lhuillier. SMPTE Journal, Jun. 1989. pp. 420-424.
"Television motion measurement for DATV and other applications" by GA Thomas. BBC Research Dept. Report Sep., 1987.
"Distorting the Time Axis: Motion-Compensated Image Processing in the Studio" by GA Thomas. International Broadcasting Convention, Brighton, UK, Sep. 23-27, 1988.
"Generation of high quality slow-motion replay using motion compensation" by GA Thomas & HYK Lau International Broadcasting Convention, Brighton, UK. Sep. 21-25, 1990.
"On the Computation of Motion from Sequences of Images-A Review" by J.K. Aggarwal and N. Nandhakumar. Proceedings of the IEEE, vo. 76, No. 8, Aug. 8, 1988, pp. 917-935.
"Motion Compensating Field Interpolation Using a Hierarchically Structured Displacement Estimator" by Matthias Bierling & Robert Thoma. Signal Processing, Vo. 11, No. 4, Dec., 1986, pp. 387-404.
"Television Standards Conversion" by Tim Borer. A thesis submitted for the degree of Doctor of Philosophy, Chapters 3,7, and 9.
"Motion Compensated Display Field Rate Upconversion" by T.J. Borer, M.G. Huyler & D.W. Parker. International Boradcasting Convention, Brighton, UK, Sep. 21-25, 1990.
"Review of motion analysis techniques" by JF Vega-Riveros & K. Jabbour. IEEE Proceedings, vol. 136, Pt. 1, No. 6, Dec., 1989, pp. 397-404.
Innovision, PLC.
Le Vu
LandOfFree
Correlation processing for motion estimation does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Correlation processing for motion estimation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Correlation processing for motion estimation will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-1597867