Speech controlled computer user interface

Details Speech controlled computer user interface Speech controlled computer user interface

1999-10-05

2001-10-30

{haeck over (S)}mits, T{overscore (a)}livaldis I. (Department: 2641)

Data processing: speech signal processing, linguistics, language

Speech signal processing

Application

C704S260000, C704S257000, C704S251000

Reexamination Certificate

active

06311159

ABSTRACT:

TECHNICAL FIELD
The present invention relates to a speech controlled computer user interface for managing communications between a user and one or more computer applications.
BACKGROUND ART
Isolation is a concept that has recently emerged in computer user interface technology. In this context, isolation refers to separating human factors and ergonomical aspects of an application (i.e., the user interface) from the functionality of the application itself. The idea of user interface builders is another important concept in this field that refers to reversing the traditional software development cycle of developing core application functionality before the user interface. Instead, the user interface is developed first. This allows human factors related to man-machine interfacing to be addressed independently of the application functionality. Many development tools are currently available that provide for convenient design of Graphical User Interfaces (GUIs) and easy integration with application software. These tools have proved to be a major step forward for GUI-based application design.
GUIs are traditionally based on a metaphor of a desktop model having various different kinds of documents. GUI applications create, modify, move, and copy these documents through point and click actions. The graphical user interface of an application include devices such as commands organized in menus and dialog boxes that contain visual controls such as buttons, sliders, text, panels, meters, etc. The desktop model metaphor has worked well with GUIs, but is not intuitive for spoken conversations.
SUMMARY OF THE INVENTION
A preferred embodiment of the present invention provides a speech controlled computer user interface for communicating between a user and at least one application program. As used herein and in the accompanying claims, “communicating between” means communications from the user to the at least one application program, communications from the at least one application program to the user, and/or both ways. The user interface includes a speech layer in communication with the user that converts between speech messages and text messages; an utterance layer in communication with the speech layer that converts between text messages and semantic meaning messages; and a discourse layer in communication with the utterance layer and the at least one application program that processes messages from the user and the at least one application program and generates responsive messages to the user and the at least one application program.
In a further embodiment, the speech layer may include at least one of: a DTMF module that converts Dial Tone Multi-Frequency (DTMF) tones into representative text-based codes; an ASR module that converts speech signals into representative text using Automatic Speech Recognition (ASR) techniques; an SMC module that converts acoustic signals into digitally encoded speech signals using Speech/Music Compression (SMC) techniques; a concatenation module that converts text messages into electronic speech representative signals; and a TTS (Text-to-Speech) module that converts text messages into representative acoustic speech signals. The utterance layer may include a natural language understanding module that converts text messages from the speech layer into representative semantic meaning messages for the discourse layer and/or a message generator module that converts semantic meaning messages from the discourse layer into representative text messages for the speech layer.
In another embodiment, the discourse layer may include a dialogue manager based on a conversational agent model that analyzes internal beliefs, intentions, and desires that are associated with the user and the at least one application, updates the beliefs, and generates new intentions. In such a case, the discourse layer may also include an application perception module that converts application messages from the at least one application program into representative beliefs for the dialogue manager, an application action module that converts intentions from the dialogue manager into representative application messages for the at least one application program, a speech perception module that converts semantic meaning messages from the utterance layer into representative beliefs for the dialogue manager, and/or a speech action module that converts intentions from the dialogue manager into representative semantic meaning messages for the utterance layer.
The dialogue manager may use a perception process that receives information from the user and the at least one application program and generates beliefs representative of current states of the user and the at least one application program. A beliefs knowledge base in communication with the perception process may contain past and current beliefs for use by the dialogue manager. A planning process in communication with the beliefs knowledge base may determine how to change a current state to attain another possible state. A desires knowledge base may contain goals for the dialogue manager to determine a desirability of alternate possible states. A commitment process in communication with the beliefs knowledge base and the desires knowledge base may compare the desirability of selected possible states and determine a desired policy based on the current state and the desirability of the selected possible states. An intentions knowledge base in communication with the commitment process may maintain intentions representative of the desired policy. An acting process in communications with the intentions knowledge base may convert the intentions into information for the user and the at least one application program to accomplish the desired policy.
A further embodiment may also include a resource manager in communication with the discourse layer that manages use of system resources by the user interface. And, a set of development tools may allow an application developer to integrate the user interface with an application program.
A preferred embodiment includes a method of communicating via a speech controlled computer user interface between a user and at least one application program. The method includes converting between speech messages and text messages with a speech layer in communication with the user; converting between text messages and semantic meaning messages with an utterance layer in communication with the speech layer; and processing messages from the user and the at least one application program with a discourse layer in communication with the utterance layer and the at least one application program, and generating responsive messages to the user and the at least one application program.
In a further embodiment, converting between speech messages and text messages may include at least one of: converting Dial Tone Multi-Frequency (DTMF) tones into representative text-based codes with a DTMF module; converting speech signals into representative text using Automatic Speech Recognition (ASR) techniques with an ASR module; converting acoustic signals into digitally encoded speech signals using Speech/Music Compression (SMC) techniques with an SMC module; converting text messages into electronic speech representative signals with a concatenation module; and converting text messages into representative acoustic speech signals with a Text-to-Speech (TTS) module. Converting between text messages and semantic meaning messages may include converting, with a natural language understanding module, text messages from the speech layer into representative semantic meaning messages for the discourse layer and/or converting, with a message generator module, semantic meaning messages from the discourse layer into representative text messages for the speech layer.
In addition, or alternatively, processing messages may include analyzing, with a dialogue manager based on a conversational agent model, internal beliefs, intentions, and desires that are associated with the user and the at least one application, updating the beliefs, and generating new intentions. Analyzi

Affiliated with

Also associated with

Rating

Speech controlled computer user interface does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Speech controlled computer user interface, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Speech controlled computer user interface will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-2572523