Data processing: measuring – calibrating – or testing – Measurement system – Measured signal processing
Reexamination Certificate
2001-05-14
2003-09-16
Hoff, Marc S. (Department: 2857)
Data processing: measuring, calibrating, or testing
Measurement system
Measured signal processing
C702S066000, C702S069000, C702S111000, C702S190000, C708S005000, C708S300000, C708S308000, C324S620000, C324S624000, C324S076190, C324S076290, C704S216000, C704S219000, C704S226000, C704S240000, C375S232000
Reexamination Certificate
active
06622117
ABSTRACT:
FIELD OF THE INVENTION
The present invention generally relates to blind source separation. More particularly, it relates to the blind separation of convolutive mixtures of mutually independent white sources.
BACKGROUND OF THE INVENTION
Blind Source Separation (BSS) addresses the issue of recovering source signals from the observation of linear mixtures of these sources. Independent Component Analysis (ICA) aims at achieving BSS in the case where the source signals are mutually statistically independent and mixed with an unknown full rank matrix, resulting in distinct instantaneous mixtures observed at the outputs of sensors. However, in most real-world situations, the assumption of instantaneous mixing between the sources does not hold, due to propagation delays and multipath propagation in reflective environments. It is more realistic to expect multiple attenuated echoes of the source signals to arrive at the sensors at different times. This situation can be modeled by assuming that the source signals are mixed with an unknown matrix of filters (instead of scalars), resulting in distinct convolutive mixtures observed at the outputs of the sensors.
The BSS of convolutive mixtures may be referred to as Convolutive ICA (CICA). So far, the algorithms devised to solve the CICA problem rely on gradient descent techniques. It is well known that gradient descent techniques are slow and that they suffer from instability problems especially because they require the empirical tuning of a learning rate.
Generally, Independent Component Analysis (ICA) aims at achieving BSS in the case where the source signals are mutually statistically independent and mixed with an unknown full rank matrix, resulting in distinct instantaneous mixtures observed at the outputs of sensors. Usually, the algorithms devised to solve the ICA problem involve estimating an unmixing matrix transforming the observed mixtures into signals that are as independent as possible. This approach is grounded in theoretical results about blind identifiability according to which, assuming that no more than one source is Gaussian, then BSS can be achieved by restoring statistical independence (see: P. Comon,
Independent component analysis, a new concept
?, Signal Processing, Special Issue on higher-order statistics, pp 287-314, vol. 36(3), 1994; X. -R. Cao and R. -W. Liu,
General approach to blind source separation
., IEEE Transactions on Signal Processing, pp 562-571, vol. 44(3), 1996; and J. -F. Cardoso,
Blind signal separation: statistical principles
, Proceedings of the IEEE, pp 2009-2025, vol.9, October 1998). As a consequence, the outputs of the unmixing matrix equal the actual sources up to scaling and permutation. In H. Attias and C. E. Schreiner,
Blind source separation and deconvolution
: the dynamic component analysis algorithm, Neural Computation, pp 1373-1424, vol. 10, 1998, an iterative procedure is proposed to learn Maximum Likelihood estimates of both the unmixing matrix and the densities of the sources, by modeling the distribution of each source with a mixture of Gaussians. Whereas the ML estimates of the density parameters are computed with the usual Expectation-Maximization equations for fitting mixtures of Gaussians, an ML estimate of the unmixing matrix is computed with gradient update rules. Recently, it was shown by S. Chen and R. Gopinath, as a by-product of their work on Gaussianization (S. S. Chen, and R. A. Gopinath.
Gaussianization
, Proceedings of the NIPS conference, 2000), that EM equations could be derived not only to compute the estimates of the density parameters, but also to compute an ML estimate of the unmixing matrix.
Convolutive ICA (CICA) aims at achieving BSS in the case where the source signals are mutually statistically independent and mixed with an unknown matrix of filters, resulting in distinct convolutive mixtures observed at the outputs of sensors. In Attias et al., supra, an iterative procedure is proposed to learn Maximum Likelihood estimates of both the unmixing filters and the densities of the sources, by modeling the distribution of each source with a mixture of Gaussians. Whereas the ML estimates of the density parameters are computed with the usual Expectation-Maximization equations for fitting mixtures of Gaussians, ML estimates of the unmixing filters are computed with gradient update rules.
In view of the foregoing, a need has been recognized in connection with overcoming the shortcomings and disadvantages presented in connection with conventional arrangements.
SUMMARY OF THE INVENTION
In accordance with at least one presently preferred embodiment of the present invention, the EM (Expectation-Maximization) solution provided for ICA is extended to the case of Convolutive ICA, where a matrix of filters, instead of a matrix of scalars, needs to be estimated. The EM equations proposed by S. Chen and R. Gopinath to solve the ICA problem is a particular case that can be addressed, where the filters would be of length 1.
Broadly contemplated herein is the replacement of gradient update rules (discussed heretofore) with Expectation-Maximization (EM) equations. The use of an EM procedure is advantageous as it provides a faster and more stable convergence than gradient descent techniques.
Addressed herein is the issue of separating convolutive mixtures of white independent sources. The model assumed by the invention is as follows. Consider D mutually independent random variables Y
1
. . . Y
D
called sources and generating real-valued data Y
1
(t) . . . Y
d
(t) . . . Y
D
(t) at each time index t. Further assume that each source Y
d
obeys a statistical law, i.e. samples are drawn according to a density probability function consisting of a mixture of l
d
Gaussians characterized by the unknown prior, mean and variance parameters {(&pgr;
d,1
, &mgr;
d,1
, &sgr;
d,1
2
)
i=1
I
d
, d=1 . . . D}. The T observations Y
d
(1) . . . Y
d
(t) . . . Y
d
(T) generated by each source Y
d
are assumed to be drawn independently, i.e. each source is assumed to be white. The D streams of data generated by the sources are not observable. They are mixed with an unknown D×D matrix of filters, resulting in D streams of data X
l
(t) . . . X
d
(t) . . . X
D
(t) which are observed at the output of D sensors. The present invention provides an EM algorithm to compute:
maximum likelihood estimates of the unknown density parameters
{(&pgr;
d,i
, &mgr;
d,i
, &sgr;
d,i
2
)
i=1
I
d
, d=
1
. . . D}
and a maximum likelihood estimate of the unknown D×D unmixing matrix of filters allowing to recover the unknown D sources (up to scaling and permutation), from the data X
1
(t) . . . X
d
(1) . . . X
D
(1) observed at the sensor outputs.
Also contemplated herein are two alternative initialization procedures of the algorithm that allow:
estimation of the length of the unmixing filters (assuming that the unmixing filters are of finite length),
speeding up of the convergence and limiting of the risk of converging towards a local maximum of the likelihood.
In summary, one aspect of the present invention provides a method of facilitating blind source separation in convolutive independent component analysis, the method comprising the steps of: initializing at least one parameter, obtaining a set comprising at least one observation; computing the likelihood of the at least one observation and of an auxiliary function; maximizing the auxiliary function; and ascertaining whether the likelihood of the at least one observation converges.
A further aspect of the present invention provides an apparatus for facilitating blind source separation in convolutive independent component analysis, the apparatus comprising: an initializing arrangement which initializes at least one parameter; an input arrangement which obtains a set comprising at least one observation; a likelihood computing arrangement which computes the likelihood of the at least one observation and of an auxiliary function; a maximizing arrangement which maximizes the auxiliary function; and an asc
Deligne Sabine
Gopinath Ramesh A.
Desta Elias
Ference & Associates
LandOfFree
EM algorithm for convolutive independent component analysis... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with EM algorithm for convolutive independent component analysis..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and EM algorithm for convolutive independent component analysis... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3092784