Data processing: speech signal processing – linguistics – language – Speech signal processing – Application
Reexamination Certificate
2000-07-07
2002-04-23
{haeck over (S)}mits, T{overscore (a)}livaldis Ivars (Department: 2654)
Data processing: speech signal processing, linguistics, language
Speech signal processing
Application
C704S235000, C704S260000, C379S052000
Reexamination Certificate
active
06377925
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates in general to an electronic translator system and methods that are particularly suited for facilitating two-way conversations or communications between speaking and/or hearing disabled individuals, and a person capable of normal communications.
2. Description of the Prior Art
In the United States, more than 28 million people, or about 10 percent of the population, have hearing impairments. These individuals often go through life without the opportunity for education, employment or the ability to lead normal lives. In the past 25 years there have been very few technological advancements related to the improvement of education or the communication skills of the deaf, hearing-impaired, blind deaf or hearing mute. Although various types of hearing aids including Cochlear implants and external FM devices have brought some degree of relief to the more profound hearing-impaired, these devices suffer from a number of shortcomings that impede their ability to assist the hearing-impaired individual in improving their communication skills.
For example, cochlear implants are intended to act as artificial cochleae by simulating the sensory hairs that a deaf person is lacking. In response to sensed sound that may come from a person's voice, the cochlear implant generates electrical impulses that stimulate the hearing receptors in the deaf person's brain. However, the devices respond not only to a person's voice, but also to all types of ambient background noise, including other people's voices, for example. As a result, the cochlear implants generate signals from numerous aural sources that the hearing impaired individual cannot distinguish from each other. A typical outcome is that a hearing impaired individual with a cochlear implant will simply turn the device off since they cannot associate the inputs generated by the implants with words spoken by a person with whom they are trying to communicate.
Other electronic devices have been developed recently that are designed to assist hearing impaired individuals in learning sign language, for example. These types of devices typically receive as input, one or more phrase or words that are entered with a keyboard, for example, and generate animations of signing hand movements that are associated with the entered words and phrase. These types of devices are certainly useful learning tools for the hearing impaired individual, but are of little value for assisting real-time communications. To date, known real-time communication aids for the hearing impaired have been limited either to the aforementioned cochlear implants and other electronic hearing aids, and devices that can convert spoken words to printed text. Speech-to-text translators, while useful for communicating with a hearing impaired individual, cannot be used to teach the hearing impaired individual the sound of the translated words as the text is generated. Until now, a need has therefore remained for a communications and learning aid for hearing impaired and other individuals that does not suffer from the shortcomings of known devices.
SUMMARY OF THE INVENTION
The present invention addresses the shortcomings of known communication and learning assistance devices for hearing impaired individuals, or the like, through provision of an electronic translator system and methods that translate input speech and text into multiple streams of data that are simultaneously delivered to a user. Preferably, the data is delivered in audible, visual and, in the case of speech input, text formats. These multiple data streams are delivered to the hearing-impaired individual in a synchronized fashion, thereby creating a cognitive response. In this manner, the present invention allows a deaf or hearing-impaired person to comprehend spoken language, to achieve two-way communications without the requirement of a human translator, to learn natural speech, to learn sign language and to learn to read. The invention is also capable of interconnection with cochlear implants and hearing aids, and alleviates the data overload caused by multiple sounds with varying volume levels being introduced to the human brain simultaneously.
To achieve the foregoing functionality, the system of the present invention, in its most preferred form, is implemented using a personal computer and various conventional peripheral devices. These include, for example, a video display monitor, keyboard, mouse, one or more audio input devices, e.g., microphone, telephone transmitter, etc. and one or more audio output devices, e.g., audio speakers, telephone receiver and hearing enhancement devices used by the deaf or hearing-impaired, such as cochlear implants and hearing aids. The computer is loaded with a number of software programs or modules that perform various translations on received speech or text. In particular, speech received by the audio input device is preferably converted to text, sign language, computer-generated speech for input to the audio devices, and animation or still images that are associated with the words (e.g., an image of an object). The sign language translation is preferably implemented by using the medium of digital movie images in which videos of a person signing words, phrase and finger spelled words are selectively accessed from a database and displayed on the video display monitor. A database of animations and images is also provided, with each of the animations or images being associated with a word or phrase in the text stream. Other databases that are also preferably provided include a phonetic spelling database to enable the phonetic spelling of words to be displayed, and an audio sound database that contains brief audio segments of environmental sounds that are associated with the words or phrase in the text stream.
The computer is preferably selected to have a fast enough processor, e.g., 500 MHz or higher, that the text, signing, audio, animation and other streams can be generated virtually simultaneously in real time, and synchronized with one another. Once synchronized, the data streams are presented to the user concurrently in a method that allows the process of mental comprehension to occur. More particularly, as the spoken words are translated into text, or the words are initially entered as text, they are simultaneously displayed on the video display monitor along with the corresponding signing videos and animation or images, and are supplied to the audio output devices. Preferably, the text words are highlighted on the screen as they are signed and spoken through the audio devices. In this manner, the user of the translator can readily associate sounds generated by a cochlear implant or a hearing aid with the signed and printed words, thus greatly improving the learning and comprehension process. In addition, the problem associated with background noise interference with the cochlear implant or hearing aid is eliminated since the generated sounds correspond only to the words of text.
Preferably, the hearing-impaired person is also able to use the system's keyboard or mouse to freely converse or respond. Words and phrases typed into the system are converted to computer-generated voice and delivered to the intended audience by the audio speakers or other means, such as the standard or cellular telephone. At this point the system has facilitated two-way conversation between a person capable of normal communications and a hearing-impaired, deaf or mute individual without the requirement of a human translator.
In addition to facilitating two-way conversation, the present invention also enables the deaf and mute to learn to speak. If an individual is physically capable of vocalizing, the invention can be used as a tool to teach linguistics. The user can utilize the computer's keyboard or mouse to generate words or phrases. The user can then practice repetition of those words or phrases while at the same time hearing a computer generated simulation of their own voice, thereby allowing
Dorety Michael F.
Greene Virginia
Greene, Jr. Morgan
Newman Harry E.
Yuhas Mark J.
Interactive Solutions, Inc.
Jones Tullar & Cooper PC
{haeck over (S)}mits T{overscore (a)}livaldis Ivars
LandOfFree
Electronic translator for assisting communications does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Electronic translator for assisting communications, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Electronic translator for assisting communications will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2825725