Data processing: speech signal processing – linguistics – language – Speech signal processing – Synthesis
Reexamination Certificate
1999-11-12
2003-12-16
Abebe, Daniel (Department: 2655)
Data processing: speech signal processing, linguistics, language
Speech signal processing
Synthesis
C704S258000
Reexamination Certificate
active
06665641
ABSTRACT:
TECHNICAL FIELD
The present invention relates to a speech synthesizer based on concatenation of digitally sampled speech units from a large database of such samples and associated phonetic, symbolic, and numeric descriptors.
BACKGROUND ART
A concatenation-based speech synthesizer uses pieces of natural speech as building blocks to reconstitute an arbitrary utterance. A database of speech units may hold speech samples taken from an inventory of pre-recorded natural speech data. Using recordings of real speech preserves some of the inherent characteristics of a real person's voice. Given a correct pronunciation, speech units can then be concatenated to form arbitrary words and sentences. An advantage of speech unit concatenation is that it is easy to produce realistic coarticulation effects, if suitable speech units are chosen. It is also appealing in terms of its simplicity, in that all knowledge concerning the synthetic message is inherent to the speech units to be concatenated. Thus, little attention needs to be paid to the modeling of articulatory movements. However speech unit concatenation has previously been limited in usefulness to the relatively restricted task of neutral spoken text with little, if any, variations in inflection.
A tailored corpus is a well-known approach to the design of a speech unit database in which a speech unit inventory is carefully designed before making the database recordings. The raw speech database then consists of carriers for the needed speech units. This approach is well-suited for a relatively small footprint speech synthesis system. The main goal is phonetic coverage of a target language, including a reasonable amount of coarticulation effects. No prosodic variation is provided by the database, and the system instead uses prosody manipulation techniques to fit the database speech units into a desired utterance.
For the construction of a tailored corpus, various different speech units have been used (see, for example, Klatt, D. H., “Review of text-to-speech conversion for English,” J. Acoust. Soc. Am. 82(3), September 1987). Initially, researchers preferred to use phonemes because only a small number of units was required-approximately forty for American English—keeping storage requirements to a minimum. However, this approach requires a great deal of attention to coarticulation effects at the boundaries between phonemes. Consequently, synthesis using phonemes requires the formulation of complex coarticulation rules.
Coarticulation problems can be minimized by choosing an alternative unit. One popular unit is the diphone, which consists of the transition from the center of one phoneme to the center of the following one. This model helps to capture transitional information between phonemes. A complete set of diphones would number approximately 1600, since there are approximately (40)
2
possible combinations of phoneme pairs. Diphone speech synthesis thus requires only a moderate amount of storage. One disadvantage of diphones is that they lead to a large number of concatenation points (one per phoneme), so that heavy reliance is placed upon an efficient smoothing algorithm, preferably in combination with a diphone boundary optimization. Traditional diphone synthesizers, such as the TTS-3000 of Lernout & Hauspie Speech And Language Products N. V., use only one candidate speech unit per diphone. Due to the limited prosodic variability, pitch and duration manipulation techniques are needed to synthesize speech messages. In addition, diphones synthesis does not always result in good output speech quality.
Syllables have the advantage that most coarticulation occurs within syllable boundaries. Thus, concatenation of syllables generally results in good quality speech. One disadvantage is the high number of syllables in a given language, requiring significant storage space. In order to minimize storage requirements while accounting for syllables, demi-syllables were introduced. These half-syllables, are obtained by splitting syllables at their vocalic nucleus. However the syllable or demi-syllable method does not guarantee easy concatenation at unit boundaries because concatenation in a voiced speech unit is always more difficult that concatenation in unvoiced speech units such as fricatives.
The demi-syllable paradigm claims that coarticulation is minimized at syllable boundaries and only simple concatenation rules are necessary. However this is not always true. The problem of coarticulation can be greatly reduced by using word-sized units, recorded in isolation with a neutral intonation. The words are then concatenated to form sentences. With this technique, it is important that the pitch and stress patterns of each word can be altered in order to give a natural sounding sentence. Word concatenation has been successfully employed in a linear predictive coding system.
Some researchers have used a mixed inventory of speech units in order to increase speech quality, e.g., using syllables, demi-syllables, diphones and suffixes (see, Hess, W. J., “Speech Synthesis—A Solved Problem, Signal processing VI: Theories and Applications,” J. Vandewalle, R. Boite, M. Moonen, A. Oosterlinck (eds.), Elsevier Science Publishers B. V., 1992).
To speed up the development of speech unit databases for concatenation synthesis, automatic synthesis unit generation systems have been developed (see, Nakajima, S., “Automatic synthesis unit generation for English speech synthesis based on multi-layered context oriented clustering,” Speech Communication 14 pp. 313-324, Elsevier Science Publishers B. V., 1994). Here the speech unit inventory is automatically derived from an analysis of an annotated database of speech—i.e. the system ‘learns’ a unit set by analyzing the database. One aspect of the implementation of such systems involves the definition of phonetic and prosodic matching functions.
A new approach to concatenation-based speech synthesis was triggered by the increase in memory and processing power of computing devices. Instead of limiting the speech unit databases to a carefully chosen set of units, it became possible to use large databases of continuous speech, use non-uniform speech units, and perform the unit selection at run-time. This type of synthesis is now generally known as corpus-based concatenative speech synthesis.
The first speech synthesizer of this kind was presented in Sagisaka, Y., “Speech synthesis by rule using an optimal selection of non-uniform synthesis units,” ICASSP-88 New York vol.1 pp. 679-682, IEEE, April 1988. It uses a speech database and a dictionary of candidate unit templates, i.e. an inventory of all phoneme sub-strings that exist in the database. This concatenation-based a synthesizer operates as follows.
(1) For an arbitrary input phoneme string, all phoneme sub-strings in a breath group are listed,
(2) All candidate phoneme sub-strings found in the synthesis unit entry dictionary are collected,
(3) Candidate phoneme sub-strings that show a high contextual similarity with the corresponding portion in the input string are retained,
(4) The most preferable synthesis unit sequence is selected mainly by evaluating the continuities (based only on the phoneme string) between unit templates,
(5) The selected synthesis units are extracted from linear predictive coding (LPC) speech samples in the database,
(6) After being lengthened or shortened according to the segmental duration calculated by the prosody control module, they are concatenated together.
Step (3) is based on an appropriateness measure—taking into account four factors: conservation of consonant-vowel transitions, conservation of vocalic sound s succession, long unit preference, overlap between selected units. The system was developed for Japanese, the speech database consisted of 5240 commonly used words.
A synthesizer that builds further on this principle is described in Hauptmann, A. G., “SpeakEZ: A first experiment in concatenation synthesis from a large corpus,” Proc. Eurospeech '93, Berlin, pp.1701-1704, 1993. The premise of this system is that if enough
Coile Bert Van
Coorman Geert
De Bock Mario
DeMoortel Jan
Deprez Filip
Abebe Daniel
Bromberg & Sunstein LLP
ScanSoft, Inc.
LandOfFree
Speech synthesis using concatenation of speech waveforms does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Speech synthesis using concatenation of speech waveforms, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Speech synthesis using concatenation of speech waveforms will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3136079