Noise padding and normalization in dynamic time warping

Details Noise padding and normalization in dynamic time warping Noise padding and normalization in dynamic time warping

1999-01-06

2002-10-15

Knepper, David D. (Department: 2654)

Data processing: speech signal processing, linguistics, language

Speech signal processing

Recognition

C704S253000, C704S244000

Reexamination Certificate

active

06466906

ABSTRACT:

FIELD OF THE INVENTION
The present invention relates to speech recognition generally and to speaker dependent recognition in the presence of noise, in particular.
BACKGROUND OF THE INVENTION
Speech recognition in noisy environments is a well studied, but yet difficult task. One such task is characterized by the following parameters:
1. The recognition is speaker dependent, where the reference templates are created from speech utterances, spoken by the user in a designated “training session”;
2. It is desired to minimize the number of training utterances to a small number (1-3), for which it is known in the art that a dynamic time warping (DTW) matching algorithm works better than a hidden markov model (HMM) algorithm;
3. The phrases to be recognized are isolated words;
4. The training phase is relatively noise-free, whereas the recognition needs to cope with additive environmental noise;
5. The environmental noise is unknown to the system prior to the instant the user pushes a push to talk (PTT) button and starts speaking;
6. The environmental noise has both stationary and non-stationary components; and
7. The system has limited fast-access memory, so that it is impossible to run DTW matching against all reference templates, in realtime and in a word-spotting manner. Therefore a two-stage processing is required, where the first stage is a voice activity detector (VAD), and the second stage is a DTW matcher.
The major difficulties imposed by the noise in the recognition phase are two:
1. Mismatch in the acoustics between the training and recognition phases; and
2. Inaccurate VAD estimates of the word endpoints in the recognition phase.
These two problems lead to recognition errors.
There are many techniques known in the art to deal with the acoustic mismatch problem. A good review can be found in Jean-Claude Junqua and Jean-Paul Haton,
Robustness in Automatic Speech Recognition
, Kluwer Academic Publishers, 1996. One technique is described in U.S. Pat. No. 5,778,342 to Erell et al.
The problem of inaccurate endpoints has been less covered in the art. One solution was given in the form of relaxed-endpoint DTW and is described in the following:
Lawrence Rabiner and Biing-Hwang Juang,
Fundamentals of Speech Recognition
, Prentice Hall, 1993;
llan D. Shallom, Raziel Haimi-Cohen and Tal Golan, “Dynamic Time Warping with Boundaries Constraint Relaxation”,
IEEE Conference in Israel,
1989
, pages 1-4; and U.S. Pat. 5,732,394 to Nakadai et al.
In normal DTW, a sequence of spectral parameters from the speech start to end point is stored as an input speech pattern. The DTW operation matches the unknown speech pattern with the content of each reference template and calculates a distance measure between them. This is performed using the graph of
FIG. 1A
, to which reference is now briefly made. The frames of the input speech pattern are placed on the X axis and those of the current reference pattern are placed on the Y axis. A path is made through the graph, starting at the lower left comer and ending at the upper right comer, where the corners are defined as the endpoints of the test and reference utterances.
However, in the relaxed-endpoint solution, shown in
FIG. 1B
to which reference is now made, the DTW paths are not constrained to start or end at the exact endpoints of the test and reference utterances. Instead, paths can start or end within a given range (delta and Qmax_delta) of the corners. This method indeed eliminates some of the errors due to inaccurate endpoints.
However, the relaxed-endpoint solutions have several disadvantages. One disadvantage is illustrated in
FIG. 2
, to which reference is now briefly made: when there exist two vocabulary words, and one word is similar to a part of the second word (this is shown by the section marked “match”), the recognition system might incorrectly indicate that utterance of the first (longer) word matches the reference template of the second (shorter) word.
Other disadvantages of the relaxed-endpoint methods are specific to the method. For example, in the article by Shallom, it is necessary to normalize, for each point on the DTW grid, the DTW accumulated score by the path length, since the relaxation of the beginning point allows now for multiple paths of different lengths. The length normalization introduces an extra computation load that does not exist in standard DTW. Also, because of the normalization, the standard DTW solution for the best matching path is in fact not optimal. For example, in U.S. Pat. No. 5,732,394, there is a higher computation load since several DTW matches are performed for each pair of test and reference patterns, instead of one.
An other solution to the problem of inaccurate endpoints is given in the following articles:
Tom Claes and Dirk Van Compemolle, “SNR-Normalization for Robust Speech Recognition”,
ICASSP
96,1996, pages 331-334;
Vijay Raman an d Vidhya Ramanujam, “Robustness Issues and Solution s in Speech Recognition Based Telephony Services”,
ICASSP
97,1997, pages 1523-1526; and
Olli Viikki and Kari Laurila, “Noise Robust HMM-Bas ed Speech Recognition Using Segmental Cepstral Feature Vector Normalization”,
ESCA
-
NATO Workshop on Robust Speech Recognition for Unknown Communication Channels
, 1997, pages 107-110.
The approach in these publications is that of a single-stage HMM-based system, running in real-time on the input speech, without a VAD. To deal with the noise segments, the HMM model of the word is concatenated on both ends with HMM model of the noise, to form a composite model of the whole utterance.
The above solution has two disadvantages: (a) This solution cannot be applied to tasks that are constrained by items (
2
) and (
7
) above; and (b) the one-pass solutions lose some of their efficiency in dealing with the acoustic mismatch (problem 1) since in one-pass algorithms there is no accurate information of the noise level. This occurs because the word endpoints are not determined prior to the recognition and therefore, the noise cannot be estimated from speech-free segments. This inaccurate noise estimate leads to recognition errors.
Another prior art method that also uses concatenated noise-speech-noise models for a DTW-based system is proposed in the article by B. Patrick Landell, Robert E. Wohlford and Lawrence G. Bahler entitled “improved Speech Recognition in Noise”,
ICASSP
86
, TOKYO
, 1986, pages 749-751. Again, the idea is to avoid the use of endpoints in the DTW matching by using noise-templates that are augmented to the speech templates and matching the whole utterance to the concatenated templates. Also, to efficiently combat the acoustic mismatch problem, it is assumed that, prior to the beginning of the utterance, the system has knowledge of the noise, so that the reference templates can be adapted to the noise prior to the beginning of the matching process.
No details are given in the Landell et al. article for how the noise templates are constructed and how to implement the DTW matching against the concatenated noise-speech-noise templates. Unlike with HMM, where the method is straightforward, this is a non-trivial problem in DTW since the DTW alignment constraints are tight but yet there is no accurate knowledge of noise template duration, since it is not known when the speaker utters the word after pushing the PTT.
Also, the Landell et al. article assumes that the noise acoustic features can be estimated from past observations of the noise, from before the speaker pushed the PTT button. For Landell et al.'s system, which was designed for an air force cockpit where the noise is fairly constant, this might be sufficient. However, with variable noise such as encountered during, for example, regular use of mobile phones, this past estimate can be inaccurate and can lead to recognition errors.
In all speech recognition applications, e. g. , in voice-dialing by name, it is very important to reject utterances that are either not in the vocabulary, or are so badly pronounced that they yield erroneous recognition. This is usually done by setting a threshold t

Affiliated with

Also associated with

Rating

Noise padding and normalization in dynamic time warping does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Noise padding and normalization in dynamic time warping, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Noise padding and normalization in dynamic time warping will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-2930423