Data processing: artificial intelligence – Neural network
Reexamination Certificate
2000-07-06
2004-02-03
Starks, Jr., Wilbert L. (Department: 2121)
Data processing: artificial intelligence
Neural network
C706S033000, C706S040000
Reexamination Certificate
active
06687686
ABSTRACT:
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH
Not applicable.
FIELD OF THE INVENTION
This invention relates generally to electrical circuits and more particularly to circuits that model the behavior of biological neurons.
BACKGROUND OF THE INVENTION
As is known in the art, there exists a class of networks referred to as neural networks which model the behavior of certain human functions. Electronic neural networks have been used to implement mathematical or engineering abstractions of biological neurons. Circuits emulating biological neurons are typically implemented using digital circuits that operate up to a million times faster than actual neurons or with software which simulates the behavior of a biological neuron. One problem with the digital circuit approach, however is that it does not utilize life-like principles of neural computation. Furthermore, a biological nervous system contains thousands or millions of interconnected neurons and thus the complexity of a biological nervous system results in a complex digital circuit.
Similarly, given the complexity of the biological systems, software simulations can take many hours or days even using presently available state-of the-art processing systems. Thus software systems are not appropriate for use in applications which require real time or close to real time performance from such systems.
An electronic circuit that emulates the analog behavior of actual biological neurons, on the other hand, can perform simulations in real time. Thus, to overcome the above limitations with systems implemented using digital circuits or software, electronic circuit neural networks which use principles of neural computation which are more life-like than the digital circuit or software approaches have been developed.
This type of neural network interacts with real-world events in a manner which is the same as or similar to biological nervous systems and can be utilized in a variety of systems including but not limited to electronic and electromechanical systems, such as artificial vision devices and robotic arms. Such neural networks can also be used as research tools to better understand how biological neural networks communicate and learn.
Much of the effort directed toward producing electronic implementations of biological neurons have focused on emulating the input-output functional characteristics of the neuron, essentially treating the neuron as an abstracted black box. These implementations focus on circuits and techniques for generating an action potential in an attempt to simulate the actions neurons take to communicate with one another. One problem with past approaches, however, is that such approaches fail to properly take into account or model the means which actually produces the action potential in a biological neuron.
Some prior art techniques have produced analog integrated circuits that mimic the functional characteristics of real neuron cells, by isomorphically emulating the membrane conductances within an actual neuron cell body. Thus, one problem with prior art approaches is that they fail to include circuitry for the synapse through which neurons communicate and/or the prior art approaches fail to include circuitry for the dendrite which is the connection between the synapse and neuron cell body. Prior art systems also fail to include effective circuitry to implement the adaptation or learning functions of real neurons.
As known to one of ordinary skill in the art, a neuromorphic system emulates the functionality and organization of a biological nervous system on an integrated circuit. Neuromorphic systems typically include analog electronic circuits with digital circuitry to enhance and support their function. Fabrication of these neuromorphic circuits is most often done in a complementary-metal-oxide-semiconductor (CMOS) process using very large-scale integration (VLSI) technology.
Neuromorphic systems directly embody in the physics of their analog CMOS circuit building blocks so-called isomorphisms of the biophysical processes. Neural computational primitives like amplifying, exponentiating, thresholding, integrating, taking the sigmoidal function of, and storing charge, can thus be efficiently performed in a real-time analog fashion using compact low-power CMOS circuits designed specifically for these purposes.
Digital computing paradigms in other fields of science and engineering has led to their use in simulating nervous systems. Digital simulations by themselves, however, require huge and complex Boolean logic functions encode fundamentally analog neural computational primitives, such as those mentioned above. Thus, the modeled system must be translated into an explicitly mathematical form. This is grossly inefficient in terms of time and number of transistors required to execute a neural computation. Significantly, the natural temporal relationship between neuronal processes is not preserved in a digital simulation on a computer thereby preventing real time interaction with the real world in a manner analogous to that of biological nervous system.
Animal nervous system are capable of learning and remembering. One simple type of learning involves the interaction of two neurons, as shown in FIG.
1
. Learning occurs when there is an alteration of the synaptic transmission strength from the presynaptic neuron's axon terminal to the postsynaptic neuron's synapse head. A synapse whose strength can be modified by neuronal activity is said to be plastic, and the general phenomenon is known as synaptic plasticity. When neuronal activity leads to an increase in synaptic transmission strength, the synapsc is said to have become potentiated. And when this stimulation leads to a decrease in strength, the synapse is said to have become depressed. If these changes are subsequently retained, the “learned” information is “remembered” by the synapse. Potentiation that is retained for a long period of time after neuronal activity has ceased is known as long-term potentiation (LTP). Likewise, depression that is retained for a long period of time after neuronal activity has ceased is known as long term depression (LTD). Both of these phenomena have been shown to occur in the various regions of the brain.
One known electrical model that attempts to explain the biophysical behavior of a Hebbian synapse is shown in FIG.
2
. Examining the simple two-neuron system of
FIG. 1
, when the presynaptic neuron fires an action potential, its axon terminal release neurotransmitters. These neurotransmitters pass through the synaptic cleft and bind to receptors on the synapse head. This causes NMDA and non-NMDA ion channels to open up. The NMDA ion channel passes an electric current, which consists primarily of Ca
2+
ions. This postsynaptic influx of Ca
2+
ions plays a pivotal role in the expression of synaptic plasticity. Upon entering the synapse head, Ca
2+
ions set in motion a series of events that ultimately leads to the induction and maintenance of LTP and/or LTD.
FIG. 3
shows how the level of calcium concentration that has accumulated inside of a synapse operates to change the long-term plasticity. The non-NMDA ion channels pass a current which, in contrast to the NMDA channels, consists mainly of Na
+
ions (with a negligible Ca
2+
component). The total synaptic current that flows through the membrane thus consists of the sum of (1) the NMDA current, and (2) the non-NMDA current; plus (3) a small leakage current, and (4) a capacitive current that flows when the head membrane voltage is changing.
The leakage conductance g
head
is constant while the non-NMDA conductance g
non-NMDA
is dependent on the time that elapses after an action potential excites the synapse. It is given by the following alpha-function
g
non
-
NMDA
(
t
)
=
κ
g
p
t
exp
(
-
t
t
p
)
where &kgr;=e/t
p
, e is the base of the natural logarithm, t
p
=1.5 ms, and the peak conductance g
p
=0.5 nS. The concentration of calcium within the synapse also modulate
Nervegna Louis
Poon Chi-Sang
Daley, Crowley & Mofford, LLP.
Massachusetts Institute of Technology
Starks, Jr. Wilbert L.
LandOfFree
Hebbian synapse circuit does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Hebbian synapse circuit, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Hebbian synapse circuit will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3295269