Television – Studio equipment
Reexamination Certificate
1999-10-13
2002-10-22
Miller, John (Department: 2614)
Television
Studio equipment
C348S901000, C348S465000, C348S700000, C348S701000, C725S014000, C725S022000
Reexamination Certificate
active
06469749
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates generally to video signal processing, and more particularly to techniques for processing video signals to identify and extract commercials or other types of video content having particular characteristics.
BACKGROUND OF THE INVENTION
Many different systems have been developed for the detection and extraction of commercials from broadcast or recorded video signals. For example, U.S. Pat. No. 4,782,401 entitled “Editing Method and Apparatus for Commercials During Video Recording” describes a hardware-oriented solution for editing out commercials in the analog domain, based on the presence of dark or blank frames used to delineate commercials.
A similar system is described in PCT Application No. WO 83/00971, entitled “Reciprocating Recording Method and Apparatus for Editing Commercial Messages from Television Signals.” This system edits out commercials based on fade-in and fade-out at the beginning and end, respectively, of a commercial break.
Another approach, described in U.S. Pat. No. 4,750,052 entitled “Apparatus and Method for Deleting Selected Program Intervals from Recorded Television Broadcasts,” utilizes a fade detector to edit commercials from a recorded broadcast program.
PCT Application No. WO 94/27404, entitled “Method and Apparatus for Classifying Patterns of Television Programs and Commercials,” uses feature extraction and a neural network to classify video signals. The system detects changes in features such as power amplitude over the frequency spectrum, color and brightness, vertical interval time code, closed caption signal, and color carrier jitter signal.
A system described in PCT Application No. WO 95/06985, entitled “Process and Device for Detecting Undesirable Video Scenes,” stores an image from a broadcast program that precedes a commercial break so that the end of the commercial break may be detected by means of the stored image. This approach makes use of the fact that broadcasters often repeat a small part of the program after the end of the commercial break.
European Patent Application No. EP 735754, entitled “Method and Apparatus for the Classification of Television Signals,” uses a set of features and associated rules to determine if the current commercials satisfy the same criteria with some degree of “fuzziness.” The set of features includes, e.g., stereo versus mono, two-channel audio, sound level, image brightness and color, and logos, used to characterize commercials. An extensive set of rules is required to accommodate thresholds and parameter variations for these features.
U.S. Pat. No. 5,708,477, entitled “Video Signal Identifier for Controlling a VCR and Television Based on the Occurrence of Commercials,” uses a video signal identifier to recognize previously-identified commercial material and to reject it either by muting the television sound and/or pausing the VCR when it is in record mode. A significant problem with this approach is that it fails to provide automatic detection, i.e., it requires the material to be identified in some way prior to its detection.
A system described in U.S. Pat. No. 5,668,917, entitled “Apparatus and Method for Detection of Unwanted Broadcast Information,” uses the repetitiveness of commercials to identify commercial material. This system stores video frames in a compressed format and compares frames in original “raw” format pixel by pixel. If the pixels match, within some threshold, then the frames are considered similar. A serious drawback of this approach is the excessive memory and computational resources that it requires. More particularly, storing video even in a compressed format takes an impractically large amount of memory space, e.g., approximately 200 GB per day for one channel of high definition television (HDTV) content. In addition, comparing raw video is very time consuming. Even assuming that compressing and decompressing video can be implemented at no additional computational cost, comparing frames will be a very slow process. A given incoming frame must be compared with the above-noted large amounts of stored video material, and the comparison completed before the next frame arrives.
As is apparent from the above, a need exists for improved techniques for identification and extraction of commercials and other types of video content, which avoid the problems associated with the above-described conventional systems.
SUMMARY OF THE INVENTION
The invention provides improved techniques for spotting, learning and extracting commercials or other particular types of video content in a video signal. In accordance with the invention, a video signal is processed to identify segments that are likely to be associated with a commercial or other particular type of video content. A signature is extracted from each of the segments so identified, and the extracted signatures are used, possibly in conjunction with additional temporal and contextual information, to determine which of the identified segments are in fact associated with the particular type of video content. The temporal information may include, e.g., an indication of the amount of time elapsed between a given signature and a matching signature from a prior segment of the video signal. The contextual information may include, e.g., program information, such as program name, channel, time slot and rating, as obtained from an electronic programming guide or other information source.
One or more of the extracted signatures may be, e.g., a visual frame signature based at least in part on a visual characteristic of a frame of the video segment, as determined using information based on DC and motion coefficients of the frame, or based on DC and AC coefficients of the frame. Other visual frame signature extraction techniques may be based at least in part on color histograms. As another example, a given extracted signature may be an audio signature based at least in part on a characteristic of an audio signal associated with at least a portion of the video segment. Other signatures in accordance with the invention include, e.g., closed caption text describing an advertised product or service, a frame number plus information from a subimage of identified text associated with the frame, such as an 800 number, a company name, a product or service name, a uniform resource locator (URL), etc., or a frame number and a position and size of a face or other object in the image, as identified by an appropriate bounding box, as well as various combinations of these and other signature types.
In accordance with another aspect of the invention, a video processing system maintains different sets of lists of signatures, the sets of lists including one or more of a set of probable lists, a set of candidate lists and a set of found lists, with each entry in a given one of the lists corresponding to a signature associated with a particular video segment. The sets of lists are updated as the various extracted signatures are processed. For example, a given one of the signatures identified as likely to be associated with the particular video content is initially placed on one of the probable lists if it does not match any signature already on one of the probable lists. If the given signature matches a signature already on one of the probable lists, the given signature is placed on one of the candidate lists. A given one of the signatures on a candidate list is moved to a found list if it matches a signature already on one of the candidate lists. A given signature may also be removed from one or more of the lists in the event that the signature is not repeated within a designated time period.
In accordance with a further aspect of the invention, the system may be configured to involve a user in the commercial spotting, learning and extraction process. For example, a user remote control for use with a television, set-top box or other video processing system may be configured to include a “never again” button, such that when the user presses that button, the commercial signature is automatically extracted and stored directly
Agnihotri Lalitha
Dimitrova Nevenka
McGee Thomas
Miller John
Natnael Paulos
LandOfFree
Automatic signature-based spotting, learning and extracting... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Automatic signature-based spotting, learning and extracting..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Automatic signature-based spotting, learning and extracting... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2990613