Data processing: speech signal processing – linguistics – language – Speech signal processing – Recognition
Reexamination Certificate
1999-10-26
2002-07-09
Dorvil, Richemond (Department: 2654)
Data processing: speech signal processing, linguistics, language
Speech signal processing
Recognition
C704S236000
Reexamination Certificate
active
06418409
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to scoring of detection systems, in general, and to error-derived scoring of detection systems, in particular.
BACKGROUND OF THE INVENTION
A detection system is a system that receives an input signal and has to detect a property of the signal, typically whether the source of the signal is a predefined object. Such systems may include radar systems, speaker recognition systems, medical diagnostic systems, fingerprint identification systems and many more. In speaker recognition systems, for example, the input signal is a collection of speech samples, and the property to be detected is the identity of the speaker. See, for example, A. Rosenberg and F. K. Soong, “Recent Research in Automatic Speaker Recognition”,
Advances in Speech Signal Processing
, S. Furui and M. M. Sondhi, Eds., Marcel Dekker Inc., 1991 and U.S. Pat. No. 5,926,555 to Ort et al.
Reference is made to
FIG. 1
, which is a block diagram illustration of a detection system, generally referenced
100
, as is known in the art. Detection system
100
comprises a detection unit (DU)
102
and an application system (AS)
104
. AS
104
issues a detection command to DU
102
with the necessary parameters. AS
104
must supply DU
102
with one or more claimed identities of the target. DU
102
collects the input signal, transforms it to some internal representation and compares the internal representation with the claimed identities. The detection result is sent to AS
104
, which performs a predefined action according to this result. In speaker recognition systems, for example, the action can be to grant the speaker access to some service, or to reject the speaker as an impostor.
In many detection systems, the detection result of DU
102
is not a binary positive
egative decision, but rather a numerical score. This score indicates the certainty of the suggested decision. A commonly used score is the posterior probability that defines the probability of detecting the searched property given the input signal, as is described in R. O. Duda and P. E. Hart,
Pattern Classification and Scene Analysis
, Wiley, 1973, pp. 10-13. In speaker recognition systems, for example, this score can be the probability that the given speech samples were generated by the claimed identity speaker.
When AS
104
has to choose an action based on the detection result, the numerical score is translated into a binary positive
egative decision. This is done by setting a threshold, so that the decision made (and the action performed) depend upon whether the score is above or below the threshold.
The threshold is set after considering many factors, such as the prior probabilities of having each result (the probabilities are known to AS
104
in advance), and the cost of the different possible errors. Two general types of errors are possible. A “false positive” error happens when a positive decision is made by AS
104
while the correct decision (by some criterion other than a comparison with the threshold) is negative. A “false negative” error happens when a negative decision is made by AS
104
while the correct decision (by some criterion other than a comparison with the threshold) is positive.
For example, an application subsystem of a speaker recognition system in a bank may decide to grant access to some data only to speakers who scored more than 0.99. The threshold is set so high because it is known that the cost of showing the data to an impostor is very high, but the cost of rejecting a true speaker is not that high because he can try again. To demonstrate the influence of prior probability on the setting of the threshold, this threshold may be increased at nighttime to 0.995 reflecting the prior knowledge that most breaking in events happen at night.
Reference is now made to
FIG. 2
, which is an illustration of a graph showing the count of scores as a function of the score value, for a hypothetical detection system, as is known in the art. A threshold
200
divides the scores into positive scores
201
(scores higher than the threshold
200
lead to a positive decision by an application system) and negative scores
202
(scores lower than the threshold
200
lead to a negative decision by the application system). A distribution
203
of scores for which the correct decision is positive, is indicated with a dashed-line curve. Similarly, a distribution
204
of scores for which the correct decision is negative, is indicated with a solid-line curve.
A hatched area
205
indicates the rate of false negative errors, i.e. scores for which the correct decision is positive and which lead to a negative decision by the application system. A hatched area
206
indicates the rate of false positive errors, i.e. scores for which the correct decision is negative and which lead to a positive decision by the application system.
Choosing the threshold
200
inaccurately may result in sub-optimal use of the detection system. If the threshold is too low, there may be too many false positive errors. The reduction of false negative errors in this case does not necessarily compensate for the increase of the false positive errors. Rather, the relative significance of false positive errors and false negative errors is expressed by the cost function. Similarly, if the threshold is too high, there may be too many false negative errors.
The calculation of the optimal threshold
200
can be very difficult, especially when the internals of the detection system are hidden from the user. On the other hand, the detection system cannot output a binary positive
egative decision when the priors of the input and the cost function are not known to the designer of the detection system.
SUMMARY OF THE INVENTION
There is provided in accordance with a preferred embodiment of the present invention a method for use in a detection unit that produces a score to be converted into a binary decision via the setting of a threshold. The method is a method for generating the score as an error-derived score such that the threshold is a tolerable one-sided error probability. The method includes the steps of generating a primary score that is a monotonic function of the posterior probability, obtaining a distribution of primary scores of input signals that ought to lead to a particular binary decision, and translating, based on the distribution, the primary score of a current input signal to the error-derived score.
In accordance with a preferred embodiment of the present invention, either the distribution is determined from knowledge of internals of the detection unit or from calibration input signals.
Moreover, in accordance with a preferred embodiment of the present invention, when the tolerable one-sided probability is a false positive error probability and the particular binary decision is a negative decision, the step of translating includes the step of determining the percentage of the input signals whose primary scores are higher than the primary score of the current input signal. Furthermore, in accordance with a preferred embodiment of the present invention, when the tolerable one-sided probability is a false negative error probability and the particular binary decision is a positive decision, the step of translating includes the step of determining the percentage of the input signals whose primary scores are lower than the primary score of the current input signal.
REFERENCES:
patent: 4489434 (1984-12-01), Moshier
patent: 5182773 (1993-01-01), Bahl et al.
patent: 5727081 (1998-03-01), Burges et al.
patent: 5926555 (1999-07-01), Ort et al.
patent: 5978495 (1999-11-01), Thomopoulos et al.
R.O. Duda and P.E. Hart,Pattern Classification and Scene Analysis, Wiley, 1973, pp. 10-13.
A. Rosenberg and F.K. Soong, “Recent Research in Automatic Speaker Recognition”,Advances in Speech Signal Processing, 1991.
Douglas A. Reynolds, “Comparison of Background Normalization Methods for Text-Independent Speaker Verification”,Proceedings of Eurospeech '97, Rhodes, Greece, pp. 963-966.
H. Gish and M. Schmidt, “Text Independent Speaker Ident
Dorvil Richemond
Eitan Pearl Latzer & Cohen-Zedek
Persay Inc.
LandOfFree
Error derived scores for detection systems does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Error derived scores for detection systems, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Error derived scores for detection systems will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2841083