Data processing: speech signal processing – linguistics – language – Speech signal processing – Application
Reexamination Certificate
1998-03-23
2001-02-06
Sax, Robert (Department: 2748)
Data processing: speech signal processing, linguistics, language
Speech signal processing
Application
C705S014270, C706S046000, C706S059000
Reexamination Certificate
active
06185534
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Technical Field
Within the human-computer interaction community there a growing consensus that traditional WIMP (windows, icons, mouse, and pointer) interfaces need to become more flexible, adaptive, and human-oriented. Simultaneously, technologies such as speech recognition, text-to-speech, video input, and advances in computer graphics are providing increasingly rich tools to construct such user interfaces. These trends are driving growing interest in agent- or character-based user interfaces exhibiting quasi-human appearance and behavior.
2. Background Discussion
One aspect of developing such a capability is the ability of the system to recognize the emotional state and personality of the user and respond appropriately. Research has shown that users respond emotionally to their computers. Emotion and personality are of interest to us primarily because of the ways in which they influence behavior, and precisely because those behaviors are communicative—in human dialogues they establish a channel of social interaction that is crucial to the smoothness and effectiveness of the conversation. In order to be an effective communicant, a computer character needs to respond appropriately to these signals from the user and should produce its own emotional signals that reinforce, rather than confuse, its intended communication.
There are two crucial issues on the path to what has been termed “affective computing”:
(1) providing a mechanism to infer the likely emotional state and personality of the user, and
(2) providing a mechanism to generate behavior in an agent (.e.g. speech and gesture) consistent with a desired personality and emotional state.
A Command and Control Agent
Imagine a diagnostic session where a user is having trouble printing and an automated, speech-enabled agent is providing assistance. The agent asks a few informational questions and then makes a suggestion “Please try the following. Go to your printer and make sure all cables are plugged in properly and the printer is turned on and is online.” The user checks this and returns, replying “No dice, it still doesn't print.” Due to the failure of the speech recognition system to recognize “dice”, the agent responds “I'm sorry, I did not understand. Please repeat yourself.” The user responds, in a some what faster and louder tone, “I said it didn't work! What should I try next?” The agent, noting the speed, volume, intonation, and wording of the utterance now has an increased probability that the user is upset, and a slightly increased belief that the person is a dominant personality. In response, the agent could decide to be either extremely submissive and apologetic for its failings so far, or respond in kind in a terse, confident fashion. The agent chooses the second path. “OK, I'm doing the best I can. Try switching the printer off and back on, and try printing again,” it replies, in a somewhat less courteous manner than the previous suggestion.
This dialogue is an example of a command and control interface, in that at each stage there are relatively few alternatives that the agent (or speech recognizer) needs to consider. In the scenario we are considering, at any point the agent need only consider responses to the previous question, as well as a few generic responses (e.g. quit). As we will see, the recognition and generation of alternative phrasings for these speech acts will provide the basis for an affective infrastructure for the agent.
A goal of the present invention is an architecture which is appropriate for a broad range of tasks that are amenable to such command and control interfaces. Such an architecture would not attempt to manipulate the probabilistic characteristics of the language model used by the speech recognition engine, but rather would interpret the various possible rephrasings of a fixed set of alternatives in terms of emotion and personality.
Bayesian Networks Employed in Carrying Out the Invention
The advent of artificial intelligence within computer science has brought an abundance of decision-support systems. Decision-support systems are computer systems in which decisions, typically rendered by humans, are recommended and sometimes made. In creating decision-support systems, computer scientists seek to provide decisions with the greatest possible accuracy. Thus, computer scientists strive to create decision-support systems that are equivalent to or more accurate than a human expert. Applications of decision-support systems include medical diagnosis, troubleshooting computer networks, or other systems wherein a decision is based upon identifiable criteria.
One of the most promising new areas for research in decision-support systems is Bayesian networks. A Bayesian network is a representation of the probabilistic relationships among distinctions about the world. Each distinction, sometimes called a variable, can take on one of a mutually exclusive and exhaustive set of possible states. A Bayesian network is expressed as an acyclic-directed graph where the variables correspond to nodes and the relationships between the nodes correspond to arcs. A simple example of a Bayesian network can have three variables, X
1
, X
2
, and X
3
, which are represented by three respective nodes with arcs connecting the nodes to reflect the various causal relationships. Associated with each variable in a Bayesian network is a set of probability distributions. Using conditional probability notation, the set of probability distributions for a variable can be denoted by p(x
i
|Π
i
, &xgr;), where “p” refers to the probability distribution, where “Π
i
” denotes the parents of variable X
i
and where “&xgr;” denotes the knowledge of the expert. The Greek letter “&xgr;” indicates that the Bayesian network reflects the knowledge of an expert in a given field. Thus, this expression reads as follows: the probability distribution for variable X
i
given the parents of X
i
and the knowledge of the expert. For example, X
1
is the parent of X
2
. The probability distributions specify the strength of the relationships between variables. For instance, if X
1
has two states (true and false), then associated with X
1
is a single probability distribution p(x
1
&xgr;) and associated with X
2
are two probability distributions p(x
2
|x
1
=t, &xgr;) and p(x
2
|x
1
=f, &xgr;).
The arcs in a Bayesian network convey dependence between nodes. When there is an arc between two nodes, the probability distribution of the first node depends upon the value of the second node when the direction of the arc points from the second node to the first node. In this case, the nodes are said to be conditionally dependent. Missing arcs in a Bayesian network convey conditional independencies. For example, two nodes may be conditionally independent given another node. However, two variables indirectly connected through intermediate variables are conditionally dependent given lack of knowledge of the values (“states”) of the intermediate variables. Therefore, if the value for the other node is known, the two nodes are conditionally dependent.
In other words, sets of variables X and Y are said to be conditionally independent, given a set of variables Z, if the probability distribution for X given Z does not depend on Y. If Z is empty, however, X and Y are said to be “independent” as opposed to conditionally independent. If X and Y are not conditionally independent, given Z, then X and Y are said to be conditionally dependent given Z.
The variables used for each node may be of different types. Specifically, variables may be of two types: discrete or continuous. A discrete variable is a variable that has a finite or countable number of states, whereas a continuous variable is a variable that has an uncountably infinite number of states. All discrete variables considered in this specification have a finite number of states. An example of a discrete variable is a Boolean variable. Such a variable can assume only one of two states: “true” or “false.”
Ball John Eugene
Breese John S.
Michaelson Peter L.
Michaelson & Wallace
Microsoft Corporation
Sax Robert
Wallace Robert M.
LandOfFree
Modeling emotion and personality in a computer user interface does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Modeling emotion and personality in a computer user interface, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Modeling emotion and personality in a computer user interface will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2574656