Data processing: speech signal processing – linguistics – language – Speech signal processing – Recognition
Reexamination Certificate
2000-11-21
2004-03-30
Dorvil, Richemond (Department: 2654)
Data processing: speech signal processing, linguistics, language
Speech signal processing
Recognition
C704S251000, C704S270100, C707S793000, C707S793000
Reexamination Certificate
active
06714909
ABSTRACT:
FIELD OF INVENTION
The invention relates to automatically performing content-based indexing on structured multimedia data
BACKGROUND OF THE INVENTION
The amount of information generated in today's society is growing exponentially. Moreover, the data is made available in more than one dimension across different media, such as video, audio, and text. This mass of multimedia information poses serious technological challenges in terms of how multimedia data can be integrated, processed, organized, and indexed in a semantically meaningful manner to facilitate effective retrieval.
When the amount of data is small, a user can retrieve desired content in a linear fashion by simply browsing the data sequentially. With the large amounts of data now available, and expected to still grow massively in the future, such linear searching is no longer feasible. One example used daily is a table of contents for a book. The larger the amount of information, the more the abstraction needed to create the table of contents. For instance, while dividing an article into a few sections may suffice, a book may need subsections or even sub-subsections for lower level details and chapters for higher level abstraction. Furthermore, when the number of books published grows rapidly, in order to assist people to choose appropriate books to buy, books are grouped into different categories such as physics, mathematics, and computer hardware or into even higher levels of abstraction such as categories of literature, science, travel, or cooking.
Usually, a content structure is designed by the producer before the data is being generated and recorded. To enable future content based retrieval, such intended semantic structure (metadata) should be conveyed simultaneously to the users as the content (data) is delivered. In this way, users can choose what they desire based on the description in such metadata. For example, every book or magazine is published together with its table of contents, through which users can find the page number (index) where the desired information is printed by simply jumping to the page.
There are different methods to generate the above described abstraction or metadata. The most intuitive one is to do it manually as in the case of books (table of contents) or broadcast news (closed caption) delivered from major American national broadcast news companies. Since manual generation of index is very labor intensive, and thus, expensive, most types of digital data in practice is still delivered without metadata attached.
SUMMARY OF THE INVENTION
The invention provides a system and method for automation of index and retrieval processes for multimedia data. The system and method provide the ability to segment multimedia data, such as news broadcasts, into retrievable units that are directly related to what users perceive as meaningful.
The method may include separating a multimedia data stream into audio, visual and text components, segmenting the audio, visual and text components based on semantic differences, identifying at least one target speaker using the audio and visual components, identifying a topic of the multimedia event using the segmented text and topic category models, generating a summary of the multimedia event based on the audio, visual and text components, the identified topic and the identified target speaker, and generating a multimedia description of the multimedia event based on the identified target speaker, the identified topic, and the generated summary.
In this regard, the method may include automatically identifying a hierarchy of different types of content. Examples of such content include different speakers (e.g., anchor), news reporting (correspondences or interviews), general news stories, topical news stories, news summaries, or commercials. From such extracted semantics, an indexed table can be constructed so that it provides a compact yet meaningful abstraction of the data. Compared with conventional linear information browsing or keywords based search with a flat layer, the indexed table facilitates non-linear browsing capability that is especially desired when the amount of information is huge.
REFERENCES:
patent: 5793903 (1998-08-01), Lopresti et al.
patent: 5852684 (1998-12-01), Lopresti et al.
patent: 5905981 (1999-05-01), Lawler
patent: 6166735 (2000-12-01), Dom et al.
patent: 6195497 (2001-02-01), Nagasaka et al.
patent: 6263507 (2001-07-01), Ahmad et al.
patent: 6317710 (2001-11-01), Huang et al.
patent: 6363380 (2002-03-01), Dimitrova
patent: 6405166 (2002-06-01), Huang et al.
patent: 6442523 (2002-08-01), Siegel
patent: 06-266495 (1994-09-01), None
patent: 08-287094 (1996-11-01), None
Özsoyo{haeck over (g)}lu et al (“Automating The Assembly Of Presentations From Multimedia Databases”, Proceedings of the Twelfth International Conference on Data Engineering, Mar. 1996).*
Magrin-Chagnolleau et al (“Indexing Telephone Conversations By Speakers Using Time-Frequency Principal Component Analysis”, IEEE International Conference on Multimedia and Expo, Jul. 2000).*
Botafogo et al “The MORENA Model For Hypermedia Authoring And Browsing”, Proceedings of the International Conference on Multimedia Computing and Systems, May 1995.).*
Not et al “ReUsing Information Repositories For Flexibly Generating Adaptive Presentations”, Conference on Information Intelligence and Systems, Nov. 1999).*
Automated Generation of News Content Hierarchy by Integrating Audio, Video, and Text Information, ICASSP, 1999, Phoenix, A Z, Mar. 1999.
Gibbon David Crawford
Huang Qian
Liu Zhu
Rosenberg Aaron Edward
Shahraray Behzad
AT&T Corp.
Dorvil Richemond
Nolan Daniel
LandOfFree
System and method for automated multimedia content indexing... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with System and method for automated multimedia content indexing..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System and method for automated multimedia content indexing... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3278706