Data processing: database and file management or data structures – Database design – Data structure types
Reexamination Certificate
1995-11-03
2001-12-18
Hong, Stephen S. (Department: 2176)
Data processing: database and file management or data structures
Database design
Data structure types
C707S793000
Reexamination Certificate
active
06332147
ABSTRACT:
FIELD OF THE INVENTION
The present invention is related to the field of multi-media capture, replay and editing of sessions.
BACKGROUND OF THE INVENTION
Much of the work of groups, even in such orderly settings as structured meetings, takes the form of casual interaction the give and take of conversational exchanges whereby a group comes to a shared understanding of the technical, process, and relational facets of their work. This casual activity is poorly supported by most computational tools, which tend to focus on the outcomes of such activity, while ignoring much of how the group arrived at those outcomes. Further, attempts to gather such information typically end up formalizing the activity, making the participants conform to a way of working that suits the information gathering tool rather than supporting their natural work practices.
Collecting audio, video, and computational recording of a meeting provides a rich, reviewable record of group processes. Unfortunately, the benefits of such a record has tended to be unwieldy, oftentimes because of the required sequential access. With the advent of digital recordings of such information the ability to instantly access such information is possible. However, instant access is only useful if you know where to go. Thus, it is necessary and desirable to index such records based on meaningful elements or events.
One approach to indexing is termed Real-Time notetaking, where a person takes notes during the meeting using a system that timestamps the various individual elements of the notes. This provides a comprehensible index into the recording of the meeting. By simply selecting any element of the notes, the system can retrieve and play the part of the AV recording that corresponds to the timestamp of the note element.
The simplest example is a textual transcription system, in which the user simply types text as the meeting proceeds (the text can be a literal transcription of what is being said or arbitrary notes about the meeting). The notes consist of a string of text, and the timestamped note elements are the individual characters of the text. After the notetaking is complete, the user can select a character and the corresponding part of the meeting recording will be retrieved and played.
Notetaking systems allow users to construct a visual representation (i.e. the notes), whose visible elements function as indices into the meeting recording. Each index element has a time associated with it, which is the time at which the user created it. Since each index element is visible, the user can select it by simply pointing to it and, since it has an associated time, obtaining an address into the meeting recording.
Notetaking systems work fine as long as the construction of the visual representation only consists of a sequence of element creating events. Problems arise when the representation is allowed to be edited during the course of the meeting, i.e. there are also element-altering and element-deleting events. The source of the problems is that the indices are the visible elements and not the events.
A downside of the notetaking approach is that a person must be devoted to the notetaking task. Thus, it would be desirable to create an index into the meeting recording without dedicating a person to the task of notetaking, and which is a by-product of the natural meeting activity itself.
Related works include:
Pedersen, E., K. McCall, T. Moran, and F. Halasz, “Tivoli: An Electronic Whiteboard for Informal Workgroup Meetings”,
Proceedings of the INTERCHI '
93
Conference on Human Factors in Computing Systems,
pp. 391-389, April 1993 which describes an electronic whiteboard application called Tivoli. The paper describes the operation of Tivoli and the motivations for its design. Several design issues are discussed including the need to reconsider the basic assumptions behind the standard desktop Graphical User Interface, the use of strokes as the fundamental object in the system, the generalized wipe interface technique, the use of meta-strokes as gestural commands.
Wolf, C., J. Rhyne, and L. Briggs, “Communications and Information Retrieval with a Pen-based Meeting Support Tool”,
Proceedings of the Conference on Computer
-
Supported Cooperative Work,
pp. 322-329, November 1992 which describes a system called We-Met (Window Environment-Meeting Enhancement Tools) for supporting communication and information retrieval needs of small group meetings. We-Met runs on workstations with an attached LCD/digitizing tablet over a local area network. We-Met provides a shared drawing area in which the participants may each work in. The shared drawing area is presented on the workstations. The reference describes the results of a user study of We-Met in group settings to better understand how it is used and a study that did not involve We-Met conducted for the purpose of better understanding how it can be used to facilitate information retrieval of recorded meeting content.
EPO Publication 0 495 612 A2 entitled “A Data Access System”, Michael G. Lamming, published Jul. 22, 1992 describes a note-taking system based on a notepad computer with an integrated audio/video recorder. As the user types on the keyboard or writes with the stylus or similar input instrument on the notepad computer, each character or stroke that is input by the user is invisibly time-stamped by the computer. This activity results in the creation of meeting “notes”. The audio/video stream is also continuously time-stamped during recording. When playback is desired, the meeting notes as finally created are presented to the user. To play a section of recording back, the user selects part of the note (perhaps by circling it with a stylus) and invokes a “playback selection” command. The computer then examines the time-stamp and “winds” the record to the corresponding place in the audio/video recording, where it starts playing—so that the user hears and/or sees what was being recorded at the instant the selected text or strokes were input. With a graphical user interface, the user may input key “topic” words and subsequently place check marks by the appropriate word as the conversation topic veers into that neighborhood.
SUMMARY
The present invention is directed to a user interface for controlling playback of temporal data representing a collaborative activity such as a meeting. Such temporal data is captured by one or more capture devices (e.g. audio recordings or an electronic whiteboard). Temporal data is comprised of timestreams and events. Timestreams are sets of timestamped data which represent some recordable aspect of the meeting. Audio and video recordings, or a stored history of the actions on an electronic whiteboard are examples of timestreams. Events are occurrences within a timestream. For example, an event may be a particularly significant action taken on the electronic whiteboard, or a change in speaker during the session. Events are used to create indices which provide direct access to a point or span in time during the collaborative activity. Timestreams may inherently define events, or alternatively, may be analyzed to identify events. Event information is comprised of a timestamp, an event type and a list of various properties of the instance of the event.
Playback of the temporal data is desirable to recall conversations and/or other activities that occurred during the meeting by both those present and by those who did not attend the collaborative activity. The timestreams of a session are played back contemporaneously. Such contemporaneous playback is desirable since it best conveys the dynamics of the meeting.
Playback of a session is performed under the control of a session access device. Coupled to the session access device are a plurality of players for playing back timestreams. The session access device utilizes event information to create a user interface for controlling session replay. The user interface is comprised of a plurality of windows. Each window may represent a player, a playback controller or an editor (which would permit a user to cre
Chiu Patrick
Kurtenbach Gordon P.
Moran Thomas P.
van Melle William J.
Domingo Richard B.
Hong Stephen S.
Xerox Corporation
LandOfFree
Computer controlled display system using a graphical replay... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Computer controlled display system using a graphical replay..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Computer controlled display system using a graphical replay... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2580622