Motion video signal processing for recording or reproducing – Local trick play processing – With randomly accessible medium
Reexamination Certificate
2000-05-16
2004-07-06
Nguyen, Huy (Department: 2615)
Motion video signal processing for recording or reproducing
Local trick play processing
With randomly accessible medium
C386S349000, C386S349000
Reexamination Certificate
active
06760536
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of Invention
The present invention relates generally to the field of digital video processing. More specifically, the present invention relates to a playback mode for digital video data. This invention has particular application to producing a content-based, variable-rate, digital video playback mode.
2. Discussion of Prior Art
In general, when discussing video content, a frame is a single, discrete video event; a shot is an unbroken sequence of frames from a single camera; and a scene is a collection of one or more adjoining shots.
Browsing video information to identify and select interesting segments of video clips has historically been performed on an edit-capable analog VCR. These analog VCRs allow video browsing and content searching usually through a fast playback function with the ability to smoothly change frame display rates.
Providing fast forward or fast backward functionality for the playback of digital video has proven to be problematic. Decoding video frames and displaying them at normal speeds is a very computationally demanding process; simply running the decoder a few times faster is not a viable solution for speeding up a digital video playback.
Some early fast forward methods include displaying every n
th
frame and skipping all intermediate frames, for an arbitrarily pre-selected n, or decoding only I frames in an MPEG encoded video. The following prior art references disclose specific attempts to address the challenges related to fast forward/reverse playback of digital video, but fail to describe a variable-rate fast digital video playback functionality which changes the speed-up rate based on the content of the video frames.
The patent to Koga et al. (4,969,039) describes a method of reducing storage space for video data by dividing a video into foreground (moving portion) and background (stationary portion) and storing the background infrequently. A varying playback rate, however, for these two different portions is not discussed.
The patent to Hatakenaka et al. (5,282,049) provides a method of recording digital video data on a DAT tape to speed-up reproduction time. The described method appears directed to reducing reproduction time when making copies, not speeding up video during playback.
The patents to Arman et al. (5,521,841 and 5,606,655) teach a video browsing method which statically displays video frames associated with scene transitions, allows a user to select a frame, and then plays the video starting from that frame.
The patent to Yeo et al. (5,708,767) provides for analyzing a video into a hierarchical graph of scenes and shots so that a user can browse the video by traversing the graph.
The patent to Chen et al. (5,751,378) describes a method of using luminance variation between frames to detect scene changes; however, varying a playback rate based on this scene information does not appear to be provided for.
The patent to Nakajima et al. (5,754,728) provides for a method for fast video player. Frames are skipped based on the amount of motion between frames, so that frames with low amounts of motion are played faster than frames with high amounts of motion. However, this reference does not consider the frame's distance from the shot first frame for changing the rate at which frames are selected. Increasing the speedup ratio along the shot is not considered.
The patent to Coleman, Jr. (5,767,923) provides for a video segment indexing method which relies on frame spatial information to detect scene cuts in video signals but fails to use this scene information to affect a video playback rate.
The patent to Lee (5,771,335) provides for a MPEG fast forward/reverse method which uses an index of all I frames to speed video transmission in a video-on-demand environment.
The article by Itzhak Wilf entitled “Computer, Retrieve for Me the Video Clip of the Winning Goal” describes a video indexing and searching method which analyzes video content to select frames of interest.
The article by John S. Borecsky and Lawrence A. Rowe entitled “Comparison of Video Shot Boundary Detection Techniques” compares and contrasts the various algorithms for detecting video scene changes.
Whatever the precise merits, features and advantages of the above cited references, none of them, individually or in combination, achieve or fulfill the purposes of the present invention. Specifically, the prior art fails to provide for a digital video frame selection method which varies the rate at which frames are selected based on the frame's content, and its position within the shot.
SUMMARY OF THE INVENTION
A video stream (e.g. MPEG, MPEG 2, etc.) is first decoded using known software/hardware technology and fed to a frame analysis application. The frame analysis application uses known algorithms, such as the ones discussed in the above-mentioned article by Borecsky and Rowe or patents 4,969,039; 5,767,923; 5,642,239; 5,778,108; and 5,245,436, to locate and mark frames which occur at shot boundaries or which satisfy other segmentation constraints. Some other possible segmentation constraints are identifying and marking frames with or without accompanying audio data and identifying and marking frames which convey advertisements as opposed to program content. The boundary-marked video is then forwarded to a frame selection method which is the focus of the present invention. Frames selected during this innovative method are then either fed to an MPEG or other video format encoder which creates a new video of just the selected frames or fed to a player that plays the selected frames as a fast forward (or fast reverse) representation of the original video.
One innovative aspect of the present invention is the selecting of frames in such a way that the resulting output video has a non-linear speed-up rate in any given shot; slower at the beginning of a shot, and increasingly faster as the shot progresses. In other words, less frames are skipped nearer the beginning of a shot than are skipped near the middle or the end of a shot. The selected frames are used to either play the original video in a fast forward/backward mode or to create a new video—a fast forward playback video summary. The present invention also contemplates the resulting output video being in the same video format as the input video or in a different format. Another use for the fast video playback is as a browsing tool of the original video. It can be used by local and remote networked users and over the Internet as a video preview because of its dramatically reduced bit rate. Also, using the video summary in a video database allows less storage space to be used and allows for quicker searching and retrieval of video clips.
REFERENCES:
patent: 4969039 (1990-11-01), Koga et al.
patent: 5282049 (1994-01-01), Hatakenaka et al.
patent: 5521841 (1996-05-01), Arman et al.
patent: 5535008 (1996-07-01), Yamagishi et al.
patent: 5606655 (1997-02-01), Arman et al.
patent: 5708767 (1998-01-01), Yeo et al.
patent: 5751378 (1998-05-01), Chen et al.
patent: 5754728 (1998-05-01), Nakajima et al.
patent: 5767923 (1998-06-01), Coleman, Jr.
patent: 5771335 (1998-06-01), Lee
patent: 5991502 (1999-11-01), Kawakami et al.
Boreczky, John S. and Lawrence A Rowe, “Comparison of Video Shot Boundary Detection Techniques,” Computer Science Division—EECS, University of California Berkeley.
Wilf, Itzhak, “Computer, Retrieve for Me the Video Clip of the Winning Goal,”Advanced Imaging, Aug. 1998, pp. 53-55.
Amir Arnon
Petkovic Dragutin
Ponceleon Dulce Beatriz
Lacasse Randy W.
Lacasse & Associates LLC
McCabe, Esq. Mark C.
Nguyen Huy
LandOfFree
Fast video playback with automatic content based variable speed does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Fast video playback with automatic content based variable speed, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Fast video playback with automatic content based variable speed will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3197111