Image analysis – Pattern recognition – Classification
Reexamination Certificate
1999-07-01
2004-04-13
Mariam, Daniel (Department: 2621)
Image analysis
Pattern recognition
Classification
C382S103000, C382S173000, C382S305000, C707S793000
Reexamination Certificate
active
06721454
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention relates to a technique for detecting and indexing semantically significant events in a video sequence.
There has been a dramatic increase in the quantity of video available to the public. This trend is expected to continue or accelerate in the future with the convergence of personal computers and digital television. To improve the value of this information to users, tools are needed to help a user navigate through the available video information and find content that is relevant. For “consumer” users, such tools should be easy to understand, easy to use, and should provide reliable and predictable behavior.
Generally, there are three categories of known content-based video indexing and retrieval systems. A first category includes methods directed to the syntactic structure of video. This category includes methods of shot, boundary detection, key frame extraction, shot clustering, tables of contents creation, video summarizing, and video skimming. These methods are generally computationally conservative and produce relatively reliable results. However, the results may not be semantically relevant since the methods do not attempt to model or estimate the meaning of the video content. As a result, searching or browsing video may be frustrating for users seeking video content of particular interest.
A second category of video indexing and retrieval systems attempts to classify video sequences into categories, such as, news, sports, action movies, close-ups, or crowds. These classifications may facilitate browsing video sequences at a coarse level but are of limited usefulness in aiding the user to find content of interest. Users often express-the object of their searches in terms of labels with more exact meanings, such as, keywords describing objects, actions, or events. Video content analysis at a finer level than available with most classification systems is desirable to more effectively aid users to find content of interest.
The third category of techniques for analyzing video content applies rules relating the content to features of a specific video domain or content subject area. For example, methods have been proposed to detect events in football games, soccer games, baseball games and basketball games. The events detected by these methods are likely to be semantically relevant to users, but these methods are heavily dependent on the specific artifacts related to the particular domain, such as editing patterns in broadcast programs. This makes it difficult to extend these methods to more general analysis of video from a broad variety of domains.
What is desired, therefore, is a method of video content analysis which is adaptable to reliably detect semantically significant events in video from a wide range of content domains.
SUMMARY OF THE INVENTION
The present invention overcomes the aforementioned drawbacks of the prior art by providing a method of detecting an event in a video comprising the steps of analyzing the content of the video; summarizing the analysis; and inferring an event from the summary. The video event detection process is, thus, decomposed into three modular levels. Visual analysis of the video content, including shot detection, texture and color analysis, and object detection occurs at the lowest level of the technique. At the second level, each shot summarized based on the results produced by the visual analysis. At the highest level of the technique, events inferred from spatial and temporal phenomena disclosed in the shot summaries. As a result, the technique of the present invention detects event which are meaningful to a video user and the technique may be extended to a broad spectrum of video domains by incorporating shot summarization and event inference modules that are, relatively, specific to the domain or subject area of the video which operate on data generated by visual analysis processes which are not domain specific.
REFERENCES:
patent: 5664227 (1997-09-01), Muldin et al.
patent: 5821945 (1998-10-01), Yeo et al.
patent: 5969755 (1999-10-01), Courtney
Yeung, et al. “Extracting Story Units from Long Programs for Video Browsing and Navigation”, IEEE, pp. 296-305, 1996.*
Qian, et al. “A Computational Approach to Semantic Event Detection”, IEEE, pp. 200-206, Jun. 1999.*
Vasconcelos, et al “Bayesian Modeling of Video Editing and Structure: Semantic Features for Video Summarization and Browsing”, IEEE Oct. 4, 1998, pp. 153-157.*
Somoliar, et al “content-based video indexing and retrieval”, IEEE, pp. 62-72, 1994.*
Maybury, et al “multimedia summaries of broadcast news”, IEEE, pp. 442-449, 1997.*
Yeo, et al “retrieving and visualizing video”, ACM, pp. 44-52, Dec. 1997.*
Gunsel, et al “hierarchical temporal video segmentation and content characterization”, SPIE, pp. 46-56, 1997.*
DeMenthon, et al “video summarization by curve simplification”, ACM, pp. 211-218, Aug. 1998.*
Ponceleon, et al “key to effective video retrieval: effective cataloging and browsing”, ACM, pp. 99-107, Aug. 1998.*
“A Robust Real-Time Face Tracking Algorithm,” by Richard J. Qian, M. Ibrahim Sezan and Kristine E. Matthews; Sharp Laboratories of America.
“Video Skimming for Quick Browsing based on Audio and Image Characterization,” by Michael A. Smith and Takeo Kanade; Carnegie Mellon University, Jul. 1995; pp. 1-21.
“Content-Based Video Browsing Tools,” by HongJiang Zhang, Stephen W. Smoliar and Jian Jhua Wu; Institute of Systems Science, National University of Singapore;SPIEvol. 2417; pp. 389-398.
“Video Visualiztion for Company Presentation and Fast Browsing of Pictorial Content,” Minerva M. Yeung, Boon-Lock Yeo;IEEE Transactions on Circuits and Systems for Video Technologyvol. 7, No. 5, Oct. 1997, pp. 771-785.
“A Spatial Logic for Symbolic Description of Image Contents,” Alberto Del Bimbo; Enrico Vicario and Daniele Zingoni;Journal of Visual Languages and Computing, 1994, pp. 267-286.
“Analysis and Presentation of Soccer Highlights from Digital Video,” Dennis Yow, et. al.; Princeton University; 1995.
“Automated Analysis and Annotation of Basketball Video,” Drew D. Saur, et. al.;SPIEvol. 3022; pp. 176-187.
“Automatic Video Indexing via Object Motion Analysis,” by Jonathan D. Courtney;Pattern Recognition; Apr., 1997, pp. 1-31.
“Indexing of Baseball Telecast for Content-based Video Retrieval,” Toshio Kawashima, et. al.,IEEE; 1998, pp. 871-874.
“Textural Features for Image Classification,” by Robert M. Haralick; K. Shanmugam and Its'hak Dinstein;IEEE; Nov., 1973, pp. 610-620.
“Models for Automatic Classification of Video Sequences,” by Giridharan Iyengar and Andrew Lippman;SPIEvol. 3312, pp. 216-227.
“A Theoretical Comparison of Texture Algorithms,” by Richard W. Conners and Charles A. Harlow;IEEEVo. PAMI-2, No. 3, May 1980, pp. 204-222.
“Texture Features for Browsing and Retrieval of Image Data,” by B.S. Manjunath and W.Y. Ma;IEEE Transactions on Pattern Analysis and Machine Intelligence; vo. 18, No. 8, Aug. 1996, pp. 837-842.
“Improved Fractal Geometry Based Texture Segmentation Technique,” by B.B. Chaudhuri, et. al.,IEE Proceedings-E; Vol. 140, No. 5, Sep. 1993, pp. 233-241.
Haering Niels C.
Qian Richard
Chernoff Vilhauer McClung & Stenzel LLP
Mariam Daniel
Sharp Laboratories of America Inc.
LandOfFree
Method for automatic extraction of semantically significant... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method for automatic extraction of semantically significant..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method for automatic extraction of semantically significant... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3211577