System for recording, annotating and indexing audio data

Details System for recording, annotating and indexing audio data System for recording, annotating and indexing audio data

1998-04-10

2002-06-11

Tsang, Fan (Department: 2645)

Telephonic communications

Audio message storage, retrieval, or synthesis

C379S088010, C379S088220, C704S270000

Reexamination Certificate

active

06404856

ABSTRACT:

BACKGROUND OF THE INVENTION
1. Field of Invention
This invention is directed to a system for recording, annotating and indexing audio data. In particular, the invention is directed to an audio system that improves the way users record, review and retrieve audio data.
2. Description of Related Art
Many people take notes on their telephone conversations. Interviews with lawyers, managers, administrative assistants, salesmen and executives have shown that certain note-taking practices are used during phone calls. Some people record the date and time of the call, as well as the identity of the caller, some people keep a separate notebook by the phone for notes on their calls and others keep a special notebook for notes taken on voice mail messages. People take notes on information obtained during their calls, generate a list of “to do” items, and record other details such as sales orders and directions. If an administrative assistant answers a call for his boss and finds the message too complex, the call is transferred to the voice mail recording system.
Recording a call increases the accuracy of the information that is obtained from the call. However, while voice mail systems can record phone conversations, they do not index the audio data for later access.
U.S. patent application Ser. No. 08/821,311, filed Mar. 20, 1997, which is incorporated herein in its entirety, discloses a system that allows users to simultaneously record audio data while taking hand-written notes. The '311 application discloses an electronic notebook that improves the way users record, review, retrieve and reuse their personal notes. The electronic notebook includes an interface that is similar to a standard paper notebook. That is, the user enters freeform hand-written notes onto a screen having the appearance of a page into a digital notebook. The electronic notebook also has features that augment standard note-taking capabilities. The electronic notebook allows a property, such as “action item” or “name” to be associated with the note marks so that notes with these properties may be more easily retrieved. In order to further aid indexing, the electronic notebook allows the user to assign text keywords to a page of notes. Finally, the electronic notebook records the audio data and synchronizes the audio data with the note marks. However, only those portions of the audio data that is highlighted by the user are permanently stored.
These features of the electronic notebook provide the ability to have multiple views on a notebook. These multiple view are analogous to views available in a database system. For example, requesting notes from last week with the property “action item” and keywords “mobile” and “computing” causes the notebook to display only those note marks from a previous week which were marked as action items and which were on the topic of mobile computing. Once a notebook's views are defined they may be reused. This provides the user with dynamic organization of the changing content of the notebook.
The note marks are linked to the audio data, so that the user can play the audio data that was recorded at the time a particular note was written. Audio data is displayed with an audio timeline at the top of the note page. Important audio data can be highlighted. Audio data highlights are shown as colored bands in the audio display that can be selected for playback. Note marks can be indexed using ink properties. Ink properties are data types that are associated with particular types of ink marks. The ink properties are indicated to the user by ink color and/or mark size. In addition, pages or note marks can be indexed by keywords. Note marks can be retrieved by queries using properties and keywords. Property queries retrieve only those note marks having the specified property, while keyword queries retrieve an entire page of note marks. The electronic notebook disclosed in the '311 application, however, does allow access to the audio data without referring to the associated note marks.
“Capturing, Structuring and Representing Ubiquitous Audio”, by D. Hindus et al., ACM Trans. Information Systems, Vol. II, No. 4, Oct. 1993, pp. 376-400, discloses a system that records telephone calls while displaying the turn-taking structure of the conversation. During the telephone call, the user selects parts of the conversation to save. After the call, the user can add text tags to the stored parts of the conversation. However, this system does not permit the user to enter pen-based input. This system only permits typed input and then only after the telephone call.
The Apple Newton® places phone calls, and provides a note page to take notes on the conversation. However, the conversation is not recorded. The Apple Newton® also records messages and sounds but does not associate properties, text or ink to the recording.
SUMMARY OF THE INVENTION
This invention provides a method and apparatus that uses keywords and properties to index audio data.
This invention further provides a method and apparatus that separates the sides of a conversation in the audio data to improve access.
This invention additionally provides a method and apparatus for recording, indexing and retrieving voice mail messages.
In particular, the method and apparatus of this invention, applies properties to the audio data and generates indexes to the audio data. This indexing allows retrieval of specific audio data and portions of audio data.
This invention allows users to take notes using a computer while recording the audio data of a telephone call. The two sides of the conversation are separated and shown in an audio display for easy access to specific comments using the associated audio data attributes, such as date and time, properties, keywords, and identity of the caller. These associated attributes are used to index the audio data for subsequent retrieval. Properties apply to the audio data as well as to note marks to provide access to specific remarks in the audio data.
This invention allows a user to play specific comments from previous calls during a current conversation. This is useful when the tone of voice used in a call conveys information that is not available in the associated note marks or text.
Properties for the audio data can be set without selecting or writing note marks by, for example, pressing a button for the desired property. The property of the audio data is displayed to the user by coloring the band for the current speaker turn in the audio bar. The method and apparatus of this invention assume that the extent of each speaker's turn corresponds to the extent of the selected audio data property.
The method and apparatus of this invention route calls based upon a mode. Depending on the mode, the method and apparatus of this invention send calls to an answering machine, accept a call for conversation, or tentatively accept a call based on the identity of the caller.
These and other features and advantages of this invention are described in or are apparent from the following detailed description of the preferred embodiments.


REFERENCES:
patent: 5390236 (1995-02-01), Klausner et al.
patent: 5526407 (1996-06-01), Russell et al.
patent: 5559875 (1996-09-01), Bieselin et al.
patent: 5651055 (1997-07-01), Argade
patent: 5655058 (1997-08-01), Balasubramanian et al.
patent: 5687221 (1997-11-01), Oheda et al.
patent: 5717879 (1998-02-01), Moran et al.
patent: 5835577 (1998-11-01), Disanto et al.
patent: 5970455 (1999-10-01), Wilcox et al.
patent: 6034970 (2000-03-01), Levac et al.
patent: 0510 411 (1992-10-01), None
patent: 0 679 005 (1995-10-01), None
patent: 0 866 397 (1998-09-01), None
patent: WO 93/07562 (1993-04-01), None
Lu, “A small Revelation: Newton has arrived—at long last”, Sep. 1993, Macworld, pp. 102-106 with an Apple advertisement for the Newton MessagePad from the same magazine, p. 29.*
Hindus, et al.,Capturing, Structuring, and Representing Ubiquitous Audio, ACM Transactoins on Information Systems, vol. 11, No. 4, Oct. 1993, pp. 376-400.
Apple Computer, Inc.Using the Notepad, Chapter 2, pp.

Affiliated with

Also associated with

Rating

System for recording, annotating and indexing audio data does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with System for recording, annotating and indexing audio data, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and System for recording, annotating and indexing audio data will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-2944140