Image analysis – Color image processing – Image segmentation using color
Reexamination Certificate
2000-07-28
2004-04-20
Mehta, Bhavesh M. (Department: 2625)
Image analysis
Color image processing
Image segmentation using color
C382S165000, C382S170000, C382S171000, C382S173000, C382S278000, C348S700000, C345S591000
Reexamination Certificate
active
06724933
ABSTRACT:
TECHNICAL FIELD
This invention generally relates to image processing and, more particularly, to a media segmentation system and related methods.
BACKGROUND
With recent improvements in processing, storage and networking technologies, many personal computing systems have the capacity to receive, process and render multimedia objects (e.g., audio, graphical and video content). One example of such computing power applied to the field of multimedia rendering, for example, is that it is now possible to “stream” video content from a remote server over a network to an appropriately configured computing system for rendering on the computing system. Many of the rendering systems provide functionality akin to that of a typical video cassette player/recorder (VCR). However, with the increased computing power comes an increased expectation by consumers for even more advanced capabilities. A prime example of just such an expectation is the ability to rapidly access relevant (i.e., of particular interest to the user) media content. Prior art systems fail to meet this expectation.
To accommodate and access the vast amount of media, a variety of image database and visual information systems have become available recently. Such systems have been used in a wide variety of applications, including medical image management, CAD/CAM systems, criminal identification systems, clip-art galleries and the like. Prior art systems may employ any of a number of search techniques to access and retrieve relevant information. By and large, such prior art systems utilize a text-based, keyword approach for indexing and retrieving such media content. In accordance with such an approach, each frame, shot or scene (each comprised of one or more of the former) is stored as a database object, wherein each image (e.g., frame, shot, scene) in the database is associated with a manually generated text description of that object. These keyword descriptors may then be searched by standard Boolean queries, where the retrieval is based on either exact or probabilistic matches of the query text.
While such prior art systems have served to whet the appetite for such technology, none of the prior art systems facilitate true content-based media searching and, thus, fail to fully address the need to accurately access and retrieve specific media content. There are several problems inherent in systems that are exclusively text-based. Automatic generation of the descriptive keywords or extraction of semantic information required to build classification hierarchies is beyond the current capability of computing vision and intelligence technologies. Consequently, the text descriptions of such images must be manually generated. It is to be appreciated that the manual input of keyword descriptors is a tedious, time-consuming process prone to inaccuracies and descriptive limitations. Moreover, certain visual properties, such as textures and patterns are often difficult, if not impossible, to adequately or accurately describe with a few textual descriptors, especially for a general-purpose indexing and retrieval applications.
While other approaches have been discussed which attempt to qualitatively segment media based on content, all are computationally expensive and, as a result, are not appropriate for near real-time consumer application. These prior art approaches typically attempt to identify similar material between frames to detect shot boundaries. Those skilled in the art will appreciate that a shot boundary often denotes an editing point, e.g., a camera fade, and not a semantic boundary. Moreover, because of the computational complexities involved, such shots are often defined as a static, or fixed number of frames preceding or succeeding an edit point (e.g., three frames prior, and three frames subsequent). In this regard, such prior art systems typically utilize a fixed window of frames to define a shot.
In contrast, scenes are comprised of semantically similar shots and, thus, may contain a number of shot boundaries. Accordingly, the prior art approaches based on visual similarity of frames between two shots often do not to produce good results, and what needed is a quantitative measure of semantic correlation between shots to identify and segment scenes.
Thus, a media segmentation system and related methods is presented, unencumbered by the inherent limitations commonly associated with prior art systems.
SUMMARY OF THE INVENTION
This invention concerns a media segmentation system and related methods, facilitating the rapid access and retrieval of media content at a semantic level. According to an exemplary implementation of the present invention, a method is presented comprising receiving media content and analyzing one or more attributes of successive shots of the received media. Based, at least in part on the analysis of the one or more attributes, generating a correlation score for each of the successive shots, wherein scene segmentation is performed to group semantically cohesive shots.
REFERENCES:
patent: 5635982 (1997-06-01), Zhang et al.
patent: 5708767 (1998-01-01), Yeo et al.
patent: 6195458 (2001-02-01), Warnick et al.
patent: 6272250 (2001-08-01), Sun et al.
patent: 6389168 (2002-05-01), Altunbasak et al.
Zheng et al, Video parsing, retrieval and browsing: an integrated and conten-based solution, Proc of 3rd ACM Internt'l Conf on Multimedia, Nov. 5-9, 1995, p 15-24.*
Zhang et al, Digital video analysis and recognition for content-based access, ACM Computing Surveys, vol 27, iss 4, Dec. 1995, p 643-644.*
Gunsel et al, Hierarchical Temporal Video Segmentation and Content Characterization, Proc SPIE, Oct. 1997, vol 3229, p 46-56.*
Corridoni et al, Structured Representation and Automatic Indexing of Movie Information Content, Pattern Recognition, 1998, vol 31, iss 12, p 2027-2045.*
Kender et al, Video scene segmentation via continuous video coherence, Proc IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Jun. 23-25, 1998, p 367-373.*
Yong Rui et al, Exploring video structure beyond shots, IEEE Internat'l Conf on Multimedia Computing and Systems, Jun. 28-Jul. 1, 1998, p. 237-240.*
Jincheng Huang et al, Integration of audio and visual information for content-based video segmentation, Proc Internat'l Conf on Image Processing, Oct. 4-7, 1998, vol 3, p 526-529.*
Wei-Ying Ma et al, Benchmarking of image features for content-based retrieval, Conf Record of the 32nd Asilomar Conf on Signals, Systems & Computers, Nov. 1-4, 1998, vol 1, p 253-257.*
Hanjalic et al, Automated high-level movie segmentation for advanced video-retrieval systems, IEEE Transactions on Circuits and Systems for Video Technology, Jun. 1999, vol 9, iss 4, p 580-588.*
Sundaram et al, Video scene segmentation using video and audio features, IEEE Internat'l Conf on Multimedia and Expo, Jul. 30-Aug. 2, 2000, vol 2, p 1145-1148.*
Hao Jiang et al, Video Segmentation with the assistance of audio content analysis, IEEE Internat'l Conf on Multimedia and Expo, Jul. 30-Aug. 2, 2000, vol 3, p 1507-1510.*
Tong Lin et al, Automatic video scene extraction by shot grouping, Proc of 15th Internat'l Conf on Pattern Recognition, Sep. 3-7, 2000, vol 4, p 39-42.*
Dictionary.com/correlation, http://dictionary.reference.com/search?q=correlation.*
Dictionary.com/semantic, http://dictionary.reference.com/search?q=semantic.*
“Video Parsing, Retrieval and Browsing: An Integrated and Content-Based Solution”, Zhang, Low, Smoliar and Wu, ACM Multimedia 95—Electronic Proceedings, Nov. 1995, pp. 1-20.
“Digital Video Analysis and Recognition for Content-Based Access”, Zhang and Tian, ACM Computing Surveys, vol. 27, No. 4, Dec. 1995.
“Automatic Video Scene Extraction by Shot Grouping”.
Lin Tong
Zhang Hong-Jiang
Lee & Hayes PLLC
Mehta Bhavesh M.
Microsoft Corporation
Sukhaphadhana Christopher
LandOfFree
Media segmentation system and related methods does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Media segmentation system and related methods, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Media segmentation system and related methods will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3275920