Data processing: speech signal processing – linguistics – language – Speech signal processing – Recognition
Reexamination Certificate
1999-11-12
2002-07-16
Dorvil, Richemond (Department: 2654)
Data processing: speech signal processing, linguistics, language
Speech signal processing
Recognition
C704S230000, C704S249000
Reexamination Certificate
active
06421641
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates generally to speech recognition systems and, more particularly, to methods and apparatus for providing fast speaker adaptation of acoustic models used in these systems.
BACKGROUND OF THE INVENTION
The performance of a general speaker-independent (SI) system is often poor for specific tasks in automatic speech recognition (ASR). Speaker adaptation techniques have recently emerged as both effective and practical methods to improve the performance of an ASR. For instance, for telephony tasks, adaptation could improve the performance significantly by compensating for the uncertainty and other effects of the telephone channel.
The most common techniques for adaptation change the acoustic models used in speech recognition using samples of speech data from a particular speaker or environment. These acoustic models attempt to model the probability density function (pdf) of acoustic feature vectors for different sounds (phones). It is common to use parametric pdf's such as gaussians to approximate the true pdf of the data. These gaussians are parametrized by means (&mgr;
i
, i=1, . . . , N) and covariances (&Sgr;
i
, i=1, . . . , N) in high dimensional (D-dimensional) feature space.
One common technique to adapt the gaussians is maximum likelihood linear regression (MLLR) adaptation. MLLR adaptation is described in C. J. Legetter and P. C. Woodland, “Maximum Likelihood Linear Regression for Speaker Adaptation of Continuous density HMM's,” Computer Speech and Language, vol. 9, no. 2, pp. 171-186, 1995, the disclosure of which is incorporated herein by reference. In this technique, the adapted gaussians that better model the speakers data are assumed to be derivable from the speaker independent gaussians by the application of an affine transform. Consequently, if &mgr;
i
represents one of the speaker independent gaussians, the speaker-adapted gaussian is assumed to be {circumflex over (&mgr;)}
i
=A&mgr;
i
where A is a D×D dimensional matrix and &mgr;
i
is a D×1 dimensional vector.
Another common technique that is used in speaker adaptation is maximum a posteriori (MAP) adaptation. MAP adaptation is described in J. L. Gauvain and C. H. Lee, “Maximum-a-Posteriori estimation for multivariate gaussian observations of Markov chains,” IEEE Trans. Speech and Audio Processing, vol. 2, no. 2, pp. 291-298, April 1994, the disclosure of which is incorporated herein by reference. Here, the features vectors in the adaptation data are assigned (with some probability) to the gaussians of the speaker independent system, and based on this assignment, the zero-th, first and second order statistics are accumulated for each gaussian. These accumulated statistics are then smoothed back with the sufficient statistics of the gaussians as computed from the training data (from which the gaussians were estimated), and the smoothed statistics are then converted into the means and covariances of the adapted gaussians.
For telephony applications, adaptation raises two concerns. First, typically, there is only as little as two seconds of data from which to adapt. This makes it imperative, especially for real-time applications, that the adaptation be fast. Second, with so little data, many of the parameters required for MLLR or MAP cannot be robustly estimated.
Since most of the telephony tasks mandate real-time processing, the SI system has to be implemented in a way that allows fast computation of gaussian likelihood values. It is to be understood that a system usually has tens of thousands of gaussians. The full computation associated with these gaussians is prohibitively expensive. One common approach to speed up the computation is to organize the gaussians in a hierarchical fashion so that only a subset of the gaussians need to be computed at any time. This is called hierarchical clustering of gaussians. Furthermore, the parameter space (means and covariances) of the gaussians is quantized. The gaussians model probability density functions in a D-dimensional feature space. These dimensions are divided into subsets of dimensions (called bands) and the gaussian's parameters (means, variances and priors) in each band are vector quantized. This is referred to as a band-quantized (BQ) system. Typically, D is 40 dimensional, there are 20 bands with 2 dimensions in each band, and the gaussians of the original system in each band are quantized into 128 bins (referred to as atoms, hence atoms represent quantized gaussians in two dimensions). The original gaussians are now represented by the closest atom in each band.
The process of computing the likelihood of a feature vector for a given gaussian is now as follows. For each band, the likelihood of the feature vector values in the band is computed for each atom in that band (this represents the computation of 128*20 2-dimensional gaussian likelihoods). Subsequently, the likelihood for the original gaussian is obtained by looking up the likelihood values of the atoms associated with the gaussian. Hence, the BQ system stores each gaussian as an index mapping (array of indices) to atoms, and also the hierarchy of mappings in the hierarchical clustering.
By way of example, the standard procedure for MLLR adaptation with the BQ system is as follows. The means (and possibly covariances) of the Dimensional gaussians are transformed into a new set of gaussians. Subsequently, based on acoustic similarity, these gaussians are clustered into hierarchical groups. Finally, for each band, the new gaussians are vector quantized to form a set of atoms. Unfortunately, this process cannot meet the real-time requirements of various applications such as, for example, telephony applications.
SUMMARY OF THE INVENTION
The present invention provides for methods and apparatus for improved speaker model adaptation with application to a speech recognition system that uses band quantized gaussians for the acoustic model. In most techniques, the speaker adapted gaussians are derived from the complete un-quantized speaker independent gaussians by some technique, and the speaker adapted gaussians are then re-quantized. The mapping of speaker independent gaussians to speaker adapted gaussians may be through an affine transformation (MLLR) or through other technique (such as the smoothing of counts, as in MAP adaptation). The present invention is applicable to all these techniques.
In the present invention, we impose certain constraints on the mapping function that are related to the division of dimensions into bands. These constraints enable the mapping function to be computed on the basis of the un-quantized gaussians, but then enable the mapping to be applied directly onto the atoms of the quantized speaker independent system, hence eliminating the overhead of having to re-quantize the adapted gaussians.
Advantageously, in accordance with the invention, by applying the adaptation directly to the atoms, the hierarchy and index mappings are left unchanged. When the mapping is properly designed with respect to the partitioning of the original feature space into bands, the computation of the transform becomes easy and hence the adaptation is fast. Additionally, since the number of adapted parameters is fairly small, the process is also more robust.
In one embodiment of the invention, the mapping function is assumed to be an affine transform, i.e., the parameters of the adapted gaussians are related to the speaker independent parameters by means of an affine transform. The application of our invention to this mapping function constrains the transformation to be block diagonal with the blocks corresponding to the dimension-bands in the system. The transforms are computed in order to maximize the likelihood of the adaptation data, or alternately some other objective function, e.g., a Bayesian likelihood, discrimination, etc., may also be used to compute the transform in accordance with the invention. It is to be understood that the adaptation data may be a short decoded sequence of speech from the speaker
Huang Jing
Padmanabhan Mukund
Dorvil Richemond
International Business Machines - Corporation
Otterstedt Paul J.
Ryan & Mason & Lewis, LLP
LandOfFree
Methods and apparatus for fast adaptation of a... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Methods and apparatus for fast adaptation of a..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Methods and apparatus for fast adaptation of a... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2884804