Data processing: artificial intelligence – Neural network – Learning task
Reexamination Certificate
2002-03-26
2003-11-04
Starks, Jr., Wilbert L. (Department: 2121)
Data processing: artificial intelligence
Neural network
Learning task
C706S031000, C706S035000
Reexamination Certificate
active
06643627
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to information processing by signal processors connected by processing junctions, and more particularly, to neural network models that simulate and extend biological neural networks.
BACKGROUND OF THE INVENTION
A biological nervous system comprises a complex network of neurons that receive and process external stimuli to produce, exchange, and store information. A neuron, in its simplest form as a basic unit for a neural network, may be described as a cell body called soma, having one or more dendrites as input terminals for receiving signals and one or more axons as output terminals for exporting signals. The soma of a neuron processes signals received from dendrites to produce at least one action signal for transmission to other neurons via axons. Some neurons have only one axon which repeatedly splits branches, thereby allowing one neuron to communicate with multiple other neurons.
One dendrite (or axon) of a neuron and one axon (or dendrite) of another neuron are connected by a biological structure called a synapse. Hence, a neural network comprises a plurality of neurons that are interconnected by synapses. Signals are exchanged and processed within such a network.
Neurons also make anatomical and functional connections with various kinds of effector cells such as muscle, gland, or sensory cells through another type of biological junctions called neuroeffector junctions. A neuron can emit a certain neurotransmitter in response to an action signal to control a connected effector cell so that the effector cell reacts accordingly in a desired way, e.g., contraction of a muscle tissue.
The structure and operations of a biological neural network are extremely complex. Many physical, biological, and chemical processes are involved. Various simplified neural models have been developed based on certain aspects of biological nervous systems. See, Bose and Liang, “Neural network fundamentals with graphs, algorithms, and applications,” McGraw-Hill (1996). A brain, for example, is a complex system and can be modeled as a neural network that processes information by the spatial and temporal pattern of neuronal activation.
One description of the operation of a general neural network is as follows. An action potential originated by a presynaptic neuron generates synaptic potentials in a postsynaptic neuron. The soma membrane of the postsynaptic neuron integrates these synaptic potentials to produce a summed potential. The soma of the postsynaptic neuron generates another action potential if the summed potential exceeds a threshold potential. This action potential then propagates through one or more axons as presynaptic potentials for other neurons that are connected. The above process forms the basis for information processing, storage, and exchange in many neural network models.
Action potentials and synaptic potentials can form certain temporal patterns or sequences as trains of spikes. The temporal intervals between potential spikes carry a significant part of the information in a neural network.
Another significant part of the information in a neural network is the spatial patterns of neuronal activation. This is determined by the spatial distribution of the neuronal activation in the network. It is desirable to stimulate both the temporal and spatial patterns in a neural network model. See, for example, Deadwyler et al., “Hippocampal ensemble activity during spatial delayed-nonmatch-to-sample performance in rats,” Journal of Neuroscience, Vol. 16, pp.354-372 (1996) and Thiels et al., “Excitatory stimulation during postsynaptic inhibition induces long-term depression in hippocampus in-vivo,” Journal of Neuroscience, Vol. 72, pp.3009-3016 (1994) and “NMDA receptor-dependent LTD in different subfields of hippocampus in vivo and in vitro,” Hippocampus, Vol. 6, pp. 43-51 (1996).
Many neural network models are based on the following two assumptions. First, synaptic strength, i.e., the efficacy of a synapse in generating a synaptic potential, is assumed to be static during a typical time scale for generating an action potential in neurons. The efficacy of a synapse is essentially a constant during a signal train. Certain models modify this assumption by allowing a slow variation over a period of processing many signal trains. In the second assumption, each sending neuron provides the same signal to all other neurons to which it is synaptically connected.
SUMMARY OF THE INVENTION
One aspect of the present invention provides an improved neural network model that removes the above two assumptions and enables neural network devices to perform complex tasks. The present invention includes information processing systems and methods that are inspired by and are configured to extend certain aspects of a biological neural network. The functions of signal processors and processing junctions connecting the signal processors correspond to biological neurons and synapses, respectively. Each of the signal processors and processing junctions may comprise any one or a combination of an optical element, an electronic device, a biological unit, or a chemical material. The processing systems and methods may also be simulated by using one or more computer programs.
Each processing junction is configured to dynamically adjust its response strength according to the temporal pattern of an incoming signal train of spikes. Hence, such a processing junction changes its response to the incoming signal and hence simulates a “dynamic synapse”.
Different processing junctions in general respond differently to the same input signal. This produces different output junction signals. This provides a specific way of transforming a temporal pattern of a signal train of spikes into a spatio-temporal pattern of junction events. In addition, the network of the signal processors and processing junctions can be trained to learn certain characteristics embedded in input signals.
One embodiment of a system for information processing includes a plurality of signal processors connected to communicate with one another and configured to produce at least one output signal in response to at least one input signal, and a plurality of processing junctions disposed to interconnect the signal processors. Each of the processing junctions receives and processes a prejunction signal from a first signal processor in the network based on at least one internal junction process to produce a junction signal which causes a postjunction signal to a second signal processor in the network. Each processing junction is configured so that the junction signal has a dynamic dependence on the prejunction signal.
At least one of the processing junctions may have another internal junction process that makes a different contribution to the junction signal than the internal junction process.
Each of the processing junctions may be connected to receive an output signal from the second signal processor and configured to adjust the internal junction process according to the output signal.
These and other aspects and advantages of the present invention will become more apparent in light of the following detailed description, the accompanying drawings, and the appended claims.
REFERENCES:
patent: 4896053 (1990-01-01), Kesselring
patent: 5214745 (1993-05-01), Sutherland
patent: 5220642 (1993-06-01), Takahashi
patent: 5222195 (1993-06-01), Alkon
patent: 5343555 (1994-08-01), Yayla et al.
patent: 5355435 (1994-10-01), DeYoung et al.
patent: 5386497 (1995-01-01), Torrey
patent: 5402522 (1995-03-01), Alkon
patent: 5408585 (1995-04-01), Burel
patent: 5553195 (1996-09-01), Meijer
patent: 5588091 (1996-12-01), Alkon
Dayhoff, Judith, Pattern recognition with a pulsed neural network, Proceedings of the conference on Analysis of neural network applications, pp. 146-159, May 1991.*
Cotter, N.E.; Mian, O.N.; A pulsed neural network capable of universal approximation, Neural Networks, IEEE Transactions on, vol.: 3 Issue: 2, Mar. 1992, pp.: 308-314.*
Maass, Wolfgang et al., Pulsed Neural Networks, The MIT Press, Nov. 30, 1998.*
Berger Theodore W.
Liaw Jim-Shih
Fish & Richardson P.C.
Starks, Jr. Wilbert L.
University of Southern California
LandOfFree
Dynamic synapse for signal processing in neural networks does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Dynamic synapse for signal processing in neural networks, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Dynamic synapse for signal processing in neural networks will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3175656