Data processing: speech signal processing – linguistics – language – Speech signal processing – Recognition
Reexamination Certificate
1999-04-29
2002-05-07
Knepper, David D. (Department: 2745)
Data processing: speech signal processing, linguistics, language
Speech signal processing
Recognition
C704S256000
Reexamination Certificate
active
06385579
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates generally to speech recognition systems and, more particularly, to methods and apparatus for forming compound words for use in speech recognition systems.
BACKGROUND OF THE INVENTION
It is an established fact that the pronunciation variability of words is greater in spontaneous, conversational speech as compared to the case of carefully read speech where the uttered words are closer to their canonical representations, i.e., baseforms. Whereas most of the speech recognition systems have focused on the latter case, there is no standard solution for dealing with the variability present in the former case. One can argue that by increasing the vocabulary of alternate pronunciations of words, i.e., acoustic vocabulary, most of the speech variability can be captured in the spontaneous case. However, an increase in the size of alternate pronunciations is typically followed by an increase in acoustic confusion between words since different words can end up having close or even identical pronunciation variants. It should be understood that the phrase “acoustic confusion” is also referred to herein as “confusability” and refers to the propensity of a speech recognition system to confuse words due to pronunciation variants.
Consider the word “TO” which when preceded by a word such as “GOING” is often pronounced as the baseform AX. That is, instead of a user uttering the phrase “GOING TO,” the user may utter the phrase “GONNA,” which may have baseforms such as G AA N AX or G AO N AX. It is well known that words may have more than one baseform since a word may be pronounced a number of ways. For instance, a vowel in a word may be pronounced as a short vowel (e.g., “A” as AX) or a long vowel (e.g., “A” as AY). Another example of the word “TO” being pronounced as AX is when the phrase “WANT TO” is uttered as “WANNA” (W AA N AX or W AO N AX).
However, in the above two examples, merely adding the baseform AX to the vocabularies of the speech recognition system for the word “TO” would lead to confusion with the word “A” for which baseform AX is the standard pronunciation.
On the other hand, most co-articulation effects, as the above two examples illustrate, arise at the boundary between adjacent words and can often be predicted based on the identity of these words. These co-articulation effects result in alterations of the last one or two phones of the first word and the first phone of the second word. These phones can undergo hard changes (e.g., substitutions or deletions) or soft changes, the latter ones being efficiently modeled by context dependent phones.
The use of crossword phonological rewriting rules was first proposed in E. P. Giachin et al., “Word Juncture Modeling Using Phonological Rules for HMM-based Continuous Speech Recognition,” Computer, Speech and Language, 5:155-168, 1991, the disclosure of which is incorporated herein by reference, and provides a systematic way of taking into account co-articulation phenomena such as geminate or plosive deletion (e.g., “WENT TO” resulting in W EH N T UW), palatization (e.g., “GOT YOU” resulting in G AO CH AX), etc.
Yet, another known way of dealing with co-articulation effects at word boundaries is to merge specific pairs of words into single compound words or multi-words and to provide special co-articulated pronunciation variants for these new tokens. For instance, frequently occurring word pairs such as “KIND OF”, “LET ME” and “LET YOU” can be viewed as single words KIND-OF, LET-ME and LET-YOU, which are often pronounced K AY N D AX, L EH M IY and “L EH CH AX,” respectively. A major reason for merging frequently co-occurring words into compound words is to tie confusable words to other words. The resulting phone sequences will be longer and therefore more likely to be recognized by the acoustic component of the speech recognition system. For instance, the word “AS” by itself is particularly confusable in spontaneous speech, but the sequence AS-SOON-AS is far more difficult to be mis-recognized.
As mentioned previously, indiscriminately adding more tokens (compound words) to the acoustic vocabulary and/or the language model will increase the confusability between words. The candidate pairs for compound words have to be chosen carefully in order to avoid this increase. Intuitively, such a pair has to meet several requirements:
1. The pair of words has to occur frequently in the training corpus. There is no gain in adding a pair with a low occurrence count (i.e., the number of times the word pair occurs in the training corpus) to the vocabulary since the chances of encountering that pair during the decoding of unseen data will be low. Besides, the compound word issued from this pair will contribute to the acoustic confusability of other words which are more probable according to the language model.
2. The words within the pair have to occur frequently together and more rarely in the pair context of other words. This requirement is necessary since one very frequent word a can be part of several different frequent pairs, e.g., (a, b
1
), . . . , (b
n+1
, a), . . . , (b
m
, a). If all these pairs were to be added to the vocabulary, then the confusability between b
i
and the pair (a, b
i
) or (b
i
, a) would be increased especially if word a has a short phone sequence. This will result in insertions or deletions of the word a when incorrectly decoding the word b
i
or the sequence b
i
−a or a−b
i
.
3. The words should ideally present co-articulation effects at the juncture, meaning that their continuous pronunciation should be different than when they are uttered in isolation. This requirement is not always compatible with the previous ones, meaning that the word pairs which have the strongest co-articulation effects do not necessarily occur very often nor do the individual words occur only together.
The use of compound words (or multi-words) was first suggested in M Finke et al., “Speaking Mode Dependent Pronunciation Modeling in Large Vocabulary Conversational Speech Recognition,” Proceedings of Eurospeech '97, Rhodos, Greece, 1997 and M. Finke, “Flexible Transcription Alignment,” 1997 IEEE Workshop on Speech Recognition and Understanding, Santa Barbara, Calif., 1997, the disclosures of which are herein incorporated by reference. These articles propose two measures for finding candidate pairs. The first measure is language model oriented and consists of maximizing the mutual information between two words while decreasing the bigram perplexity of the augmented language model (by these tokens) on the training corpus. It is to be appreciated that perplexity measures the quality of the language model and may be represented as:
Perp
=
ⅇ
-
1
N
log
P
(
C
)
,
(
1
)
where C represents the training corpus, N represents the number of words in corpus C, and P(C) represents the probability of the corpus C according to the language model. A low perplexity value translates to a better language model. Bigram perplexity refers to the perplexity of a language model with bigram probabilities and may be represented as:
P
(
C
)
=
∏
t
=
1
N
P
(
w
i
❘
w
i
-
1
)
,
(
2
)
where C=w
1
, . . . , W
N
. Even though the mutual information between two words is a good measure with respect to the second requirement above, the minimization of the perplexity is in contradiction with the first (and most important) requirement. Indeed, according to the first requirement, frequent pairs are good candidates for compound words. The language model, built by adjoining these pairs to the existing vocabulary, will present a higher perplexity on the training data since the bigram count for every pair (which was high) has been taken out by merging the pair into a single (compound) word. In other words, the prediction power of the language model without compound words is stronger (or the perplexity lower) because it can often predict the second word given the first word of a frequently seen pair. It may be readily shown that
Padmanabhan Mukund
Saon George Andrei
International Business Machines - Corporation
Knepper David D.
Otterstedt Paul J.
Ryan & Mason & Lewis, LLP
LandOfFree
Methods and apparatus for forming compound words for use in... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Methods and apparatus for forming compound words for use in..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Methods and apparatus for forming compound words for use in... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2818999