Television – Image signal processing circuitry specific to television – Special effects
Reexamination Certificate
2000-10-31
2004-06-22
Lee, Michael H. (Department: 2614)
Television
Image signal processing circuitry specific to television
Special effects
C348S598000, C348S599000
Reexamination Certificate
active
06753929
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates generally to processing video images and, in particular, to picture segmentation and superposition of real time motion pictures.
BACKGROUND OF THE INVENTION
The problem of combining parts from different images to create a new superpositioned picture can be decomposed into the following steps: picture segmentation, positioning and scaling, light and spatial parameter matching and superposition of video image parts. As is well known, picture segmentation is a formidable problem.
Picture segmentation involves separating the picture part of interest (PPI) from other parts of an image. These latter parts are called the background (BG). The separated picture part of interest is then embedded into another picture, called the basic picture (BP).
At present, picture part of interest separation from a background is based on either chroma-key, luma-key or code key methods. However, these background keys generally require specific room, studio or light conditions or a priori information about the position of the PPI in the given frame. Therefore, their use is unsuitable for many applications where these standardized conditions can not be or are usually difficult to meet.
Much research has been carried out trying to improve picture segmentation. The following patents discuss various aspects of prior art methods in this field:
U.S. Pat. Nos. 3,961,133, 5,301,016; 5,491,517; 5,566,251; Japanese Patents 4-83480, 6-133221, 55-71363 and Great Britain Patent 1,503,612.
The following articles and books also deal with the subject:
Richard Brice,
Multimedia and Virtual Reality Engineering
, 1997, Newnes, 307
Lynn Conway and Charles J. Cohen, “Video Mirroring and Iconic Gestures: Enhancing Basic Videophones to Provide Visual Coaching and Visual Control”,
IEEE Transactions On Consumer Electronics
, vol. 44, No. 2, p. 388-397, May 1998;
Andrew G. Tescher, “Multimedia is the Message”,
IEEE Signal Processing Magazine, vol.
16, No 1., pp. 44-54, January 1999; and
Eugene Leonard, “Considerations Regarding The Use Of Digital Data To Generate Video Backgrounds”,
SMPTE Journal
, vol. 87, pp. 499-504, August 1978,
SUMMARY OF THE INVENTION
The present invention relates to processing video images and, in particular, to picture segmentation and superposition of real time motion pictures. The images processed can be in arbitrary, non-standardized backgrounds. The images can be video images with the picture part of interest moving in and out of focus. In some embodiments, the background used for superposition generally are still images, while in others the background can be moving images. The system can operate in real time with the signals being processed pixel-by-pixel, line-by-line, and frame-by-frame. There are no processing interruptions and video signal loss.
The present invention describes a method for image processing of a frame. The method includes the step of separating a picture part of interest from the frame, where the frame has an arbitrary background. In another embodiment, the above separating step may further include the steps of receiving a background frame having an arbitrary background, receiving an input frame having a picture part of interest within an arbitrary background, and separating the picture part of interest from the arbitrary background using the input and background frames.
The method of the invention may use a background frame that is a still or moving image. The method may include a background frame or a picture part of interest that is out of focus.
In an embodiment of the present invention, the step of separating includes the step of spatially separating the picture part of interest and background of the input frame.
In another embodiment of the invention, the step separating further includes the step of generating the difference between luminance and chrominance signal values of the input and background frames.
According to another embodiment, the step of separating further comprises the steps of filtering an input signal, estimating the pulse signal maximum of the filtered input signal, and determining the time difference between adjacent pulse signal maxima of the input signal. This is followed by comparing the time difference to a threshold value, and accepting a signal for use as part of a picture part of interest mask if the signal is below the threshold value. The above steps generally are applied separately to the luminance, red chrominance and blue chrominance components of the input signal.
According to one aspect of the invention, a system is taught for separating a picture part of interest of an input frame. The input frame has an arbitrary background. The system comprises a mask generating unit for generating a picture part of interest mask using the difference between chrominance and luminance signal values in the input frame and an arbitrary background frame. The system also includes a separator unit for separating a picture part of interest from the input frame using the picture part of interest mask.
In another embodiment, the mask generating unit further comprises a luminance compensation unit for compensating for changes in background luminance signal when going from frame to frame. The unit also includes a means for generating a picture part of interest by removing the compensated background luminance signal from a picture part of interest in an arbitrary background signal.
In yet another embodiment of the present invention, the above mask generating unit further comprises a means for generating a chrominance signal for a picture part of interest by subtracting out a background chrominance signal from the picture part of interest in an arbitrary background signal.
According to another embodiment, the above mask generating unit further comprises a background luminance frame memory for providing a background luminance signal, a first divider for dividing luminance signals from an input frame and a memorized background luminance signal (quotient A), the memorized background luminance being provided by the background luminance frame memory. The mask generating unit also includes a second switch receiving a luminance window from a window pulse generator unit and quotient A from the first divider. The mask generating unit further includes an averaging circuit for calculating an estimated light change coefficient K by averaging quotient A over the pixels of the luminance window. There is also a second divider for detecting changes in the input frame luminance signal by dividing an input frame luminance signal by the light change coefficient K (quotient B), and a first summer for subtracting the quotient B from the memorized background luminance signal.
In still another embodiment the mask generating unit further comprises a background chrominance frame memory for providing a memorized background chrominance ratio signal, the memory being set up by a switch. It also includes a divider for dividing red and blue chrominance signals (quotient C), the chrominance signals received from a color unit, and a summer for subtracting the quotient C from the memorized background chrominance ratio signal.
In yet another embodiment of the present invention, the separator unit of the system comprises a time aligner for aligning color signals of the input frame, and a switch receiving the time aligned input frame color signals, and a picture part of interest mask from the mask generating unit. The switch generates a picture part of interest from the mask and the color signals.
Another aspect of the invention teaches a system for separating a picture part of interest from an input frame having an arbitrary background where the system includes at least one high pass filter for determining the edges of an input signal of the picture part of interest within the arbitrary background, at least one pulse signal maximum estimator for determining the time maxima of the filtered input signal, and at least one maximum-to-maximum time determiner for determining the time difference between consecutive signal maxima. The system also incl
Sheraizin Semion M.
Sheraizin Vitaly S.
Eitan, Pearl, Latzer & Cohen Zedek LLP
Lee Michael H.
VLS Com Ltd.
LandOfFree
Method and system for real time motion picture segmentation... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and system for real time motion picture segmentation..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and system for real time motion picture segmentation... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3343576