Data processing: speech signal processing – linguistics – language – Speech signal processing – Recognition
Reexamination Certificate
1999-10-20
2004-02-10
Dorvil, Richemond (Department: 2654)
Data processing: speech signal processing, linguistics, language
Speech signal processing
Recognition
C704S255000
Reexamination Certificate
active
06691088
ABSTRACT:
FIELD OF THE INVENTION
The invention relates to a method of determining parameters of a statistical language model for automatic speech recognition systems by means of a training corpus.
Automatic speech recognition systems based on large vocabularies and used, for example, for dictation systems utilize on the one hand acoustic models and on the other hand language models, which models are interrelated by means of the Bayes formula. Acoustic modeling is based on so-called HMM (“Hidden Markov Models”). Parameter values of the language model which were determined from the frequencies of occurrence (so-called “counts”) in the training corpus and which represent probability values are assigned to single vocabulary elements such as words or to sequences of vocabulary elements (so-called n-grams) such as, for example, bigrams (n=2) and trigrams (n=3) in the language modeling process.
DESCRIPTION OF PRIOR ART
It is known from T. R. Niesler and P. C. Woodland, “Combination of word-based and category-based language models”, Proc. ICSLP, vol. 1, pp. 220-223, October 1996, to base a language model on n-grams of different lengths n with corresponding conditional probabilities, wherein either a sequence of words or a sequence of categories is used as the history for a word of the vocabulary, and a category embraces a plurality of different words of the vocabulary each time. In the cases with category-based histories, the conditional probability for said category derived through evaluation of the relevant frequency of occurrence (count) is multiplied by a weighting factor corresponding to the probability of the word within said category.
SUMMARY OF THE INVENTION
One object of the invention is to modify the process of speech modeling such that the perplexity and the error rate in the speech recognition are improved.
This object is achieved in that at least a proportion of the elements of a vocabulary used is combined so as to form context-independent vocabulary element categories, in that the frequencies of occurrence of vocabulary element sequences, and if applicable the frequencies of occurrence of derived sequences formed from said vocabulary element sequences through the replacement of at least one vocabulary element by the associated vocabulary element class, are evaluated in the language modeling progress, and in that the parameters of the language model are derived from the frequencies of occurrence thus determined.
In such a process of estimating language models, the sequences of vocabulary elements and/or vocabulary element classes can be optimally attuned to the vocabulary and training material used at any time. In particular, one vocabulary element represents one word each time. The use of vocabulary element classes has the result that probability values can be better estimated, and that a smaller training corpus suffices compared with a language modeling process without the use of vocabulary element classes for achieving equally good perplexity values and error rates. Any vocabulary element of a vocabulary element sequence may be replaced by the associated vocabulary element category, if applicable. The required memory space is also reduced. An alternative or a corrective measure based on pure vocabulary element sequences is always available in the method according to the invention for those vocabulary elements for which a description by means of a category assignation is less suitable.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
In an embodiment of the invention, both the frequency of occurrence of the vocabulary element sequence and the frequencies of occurrence of the derived sequences are used for forming a language model parameter from a sequence of vocabulary elements and associated derived sequences which each comprise at least one vocabulary element category. All sequences of vocabulary elements or vocabulary element classes formed for determining a language model parameter in this embodiment take part in the formation of this language model parameter. Preferably, an average value is formed for the language model parameters of the respective sequences of vocabulary elements or vocabulary element categories.
A further embodiment may be characterized in that, for the formation of a language model parameter from a sequence of vocabulary elements and associated derived sequences which each comprise at least one vocabulary element class, exclusively the frequency of occurrence of one of said sequences is utilized. A possible selection criterion is a criterion oriented towards minimizing the perplexity.
The invention also relates to an automatic speech recognition system with a statistical language model formed in accordance with the method according to the invention.
REFERENCES:
patent: 5745876 (1998-04-01), Kneser et al.
patent: 6009392 (1999-12-01), Kanevsky et al.
patent: 6081779 (2000-06-01), Besling et al.
patent: 6125345 (2000-09-01), Modi et al.
patent: 6157912 (2000-12-01), Kneser et al.
patent: 6208971 (2001-03-01), Bellegarda et al.
P. Kneser, “Statistical language modeling using a variable context length,” International Conference on Spoken Language Proceedings, ICSLP 96, Oct. 1996, vol. 1, pp. 494 to 497.*
Kneser et al., “Improved backing-off for M-gram language modeling,” International Conference on Acoustics, Speech, and Signal Processing, ICASSP-95, May 1995, vol. 1, pp. 181 to 184.*
Kneser et al., “Semantic clustering for adaptive language modeling,” IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP-97, Apr. 1997, vol. 2, pp. 779 to 782.*
Combination of Word-Based and Category Based Language Models, T.R. Niesler et al, Proc. ICSLP, vol. 1, pp. 220-223, Oct. 1996.
Dorvil Richemond
Koninklijke Philips Electronics , N.V.
Lerner Martin
LandOfFree
Method of determining parameters of a statistical language... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method of determining parameters of a statistical language..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method of determining parameters of a statistical language... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3292698