Television – Image signal processing circuitry specific to television – Motion dependent key signal generation or scene change...
Reexamination Certificate
1999-03-18
2002-12-10
Diep, Nhon (Department: 2613)
Television
Image signal processing circuitry specific to television
Motion dependent key signal generation or scene change...
C382S173000
Reexamination Certificate
active
06493042
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of Invention
This invention relates generally to video segmentation. More particularly, this invention is directed towards systems and methods that robustly detect fades and/or dissolves in video sequences.
2. Description of Related Art
Major techniques that have been used in video segmentation can be summarized as histogram comparison, pixel-wise intensity comparison, motion-based comparison, compression factor or difference comparison, and edge fraction comparison.
The simplest way to detect whether two frames are significantly different from each other is to compare corresponding pixels in the two frames. The number of pixels in which the difference between that pixel and the spatially corresponding pixel in a second frame is greater than a threshold provides a measure of the distance between the two frames. A major drawback of these methods is their sensitivity to moving objects and movements of the camera. For example, camera panning or a large change in a small area of the frame may yield false detections.
Methods using intensity histograms seem to perform better than these pairwise comparison techniques. There are two basic techniques in this category: global histogram comparison, where a boundary is declared when the difference between histograms of two consecutive frames exceeds a threshold; and local histogram comparison, where a frame is first divided into non-overlapping regions, then the histogram of each individual region is determined. However, these methods can miss cuts when scenes look quite different but have similar distributions.
Motion-based comparison methods are based on the ratio of velocity, i.e., rate of change of motion, to the motion in each frame of the video, and an average interframe correlation coefficient. For motion-based comparison, boundary points are defined as the points where the correlation between frames is low and the ratio of velocity to motion is high.
For a feature-based technique to segment the video, the percentage of edges that enter or exit between the consecutive frames is determined and used to detect the boundaries. However, this technique requires estimating the motion beforehand to align the consecutive frames. The success of the segment detection thus depends on the robustness of the motion estimation technique.
SUMMARY OF THE INVENTION
While these conventional methods provide reasonable results for discontinuous boundaries, they are not robust in detecting boundaries that are due to dissolves, wipes, fade-ins, fade-outs, and continuous cuts.
Video segmentation is a first step towards automatic annotation of digital video sequences. Segmenting the video source involves detecting boundaries between uninterrupted segments of the video, e.g., scene breaks, by evaluating a function S(I
k
, I
k+1
) of two consecutive frames I
k
and I
k+1
. Within a video sequence, consecutive frames may differ from each other due to object and/or camera movement, lighting changes, gradual changes, e.g., continuous cuts, or abrupt changes, e.g., camera cuts and/or discontinuous cuts. Any scene change detection method should be sensitive to gradual changes and abrupt changes but should also ignore object and camera movement, as well as lighting changes, as much as possible. One method to achieve this is to perform a large spatial averaging on frame data prior to scene change detection. However, this makes it difficult to detect both the abrupt changes and the gradual changes at the same time.
This invention provides methods and systems that enable feature-based hierarchical video segmentation.
This invention separately provides methods and systems that segment a video.
This invention additionally provides methods and systems that use a two-pass procedure to detect scene breaks in video sequences.
The methods and systems of this invention provide robust systems and methods that detect both continuous and discontinuous cuts simultaneously.
The methods and systems of this invention use a two-pass procedure that detects the camera cuts on low resolution images and detects the gradual changes on high resolution images. For camera cut detection, a histogram comparison technique is used due to its robustness. The gradual changes are detected by examining the frames between the previously-detected discontinuous cuts using an intensity comparison and frame structure technique.
These and other features and advantages of this invention are described in or are apparent from the following detailed description of the preformed embodiments.
REFERENCES:
patent: 5635982 (1997-06-01), Zhang et al.
patent: 5642294 (1997-06-01), Taniguchi et al.
patent: 5767922 (1998-06-01), Zabih et al.
patent: 5835163 (1998-11-01), Liou et al.
patent: 5990980 (1999-11-01), Golin
patent: 6061471 (2000-05-01), Coleman, Jr.
patent: 6072542 (2000-06-01), Wilcox et al.
patent: 6195458 (2001-02-01), Warnick et al.
H. Yu et al., “Feature-Based Hierarchical Video Segmentation,” 1996.
Bozdagi Gozde
Bryll Robert
Harrington Steven J.
Yu Hong Heather
Diep Nhon
Oliff & Berridg,e PLC
Palazzo Eugene
Xerox Corporation
LandOfFree
Feature based hierarchical video segmentation does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Feature based hierarchical video segmentation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Feature based hierarchical video segmentation will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2959479