Data processing: artificial intelligence – Neural network – Structure
Reexamination Certificate
2000-01-10
2002-10-22
Davis, George B. (Department: 2121)
Data processing: artificial intelligence
Neural network
Structure
C326S035000
Reexamination Certificate
active
06470328
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates to hardware implementation of artificial neuron, specifically to a synapse element, a threshold circuit or a neuron circuit using the synapse elements, and a learnable neuron device.
2. Description of the Related Art
The highly developed Neumann-type computer has a far better performance in numerical calculation compared with the human ability. This type of computer still requires vast calculating time in pattern recognition or image processing, and it is insufficient especially in information processing such as association, memorization and learning which are easily performed by a human brain. There is an approach to utilize neural networks constructed by simulating functions of the human brain for providing the computer calculating facilities to perform the above processings.
The human brain has highly sophisticated functions though its basic construction is rather simple. The brain consists of nerve cells called neurons having calculating functions and nerve fibers propagating the calculated results to the other neurons. A neuron connects to nerve fibers by so-called synapse links, and the synapse links provide the neuron with the signals propagated through each nerve fiber after modifying them each with a proper weight function. The neuron is stimulated to provide an output signal to the nerve fibers connected to other neurons, which phenomenon is called fire, only when the total sum of the signals input through the nerve fibers exceeds a certain threshold value, while the neuron does not produce any output signal when the total sum is lower than the threshold value.
The human brain having a six-layer structure of a vast number of neurons connected each other performs complex information processings. The human learning process can be recognized as a process for changing the weights in the synapse links.
The neural networks are calculation structures constructed by complied layers in multiple stages formed of a vast number of elements having neural function connected each other modeling the nervous network of human brain. Therefore, the neural networks are expected to perform more easily high-degree processings of information such as pattern recognition, image processing, association, memorization, learning, and so on.
The development of semiconductor technology in recent years brings possibility to provide physical neural networks based on their conceptional designs.
Today's and especially future networks require or can require hundreds or even thousands of neurons with hundreds of input synapses each. Saving just one transistor saves tens and hundreds of thousands transistors.
A threshold element (TE) has been commonly studied because it is the simplest model of the neuron. A threshold element is proposed to be constructed by &ngr;MOS circuits having analogue amplifiers, output wired inverters, or floating gates.
The critical parameter is the permissible sum of the input weights and threshold which depends on possible variations of technological and physical parameters. In the learnable neurons, the parameter variations are compensated during the learning and the critical parameter becomes the sensitivity of the output amplifier that, in fact, is of the same order for most available implementations. Hence, the main criterion for choosing the basic TE when implementing a learnable artificial neuron should be the number of transistors per one synapse.
In the conventional semiconductor technologies, a vast number of semi-conductors should be combined to realize the functions in only one neuron. Even a limited number of semiconductors required in a restricted practical use cannot be integrated in a single semiconductor chip, so that construction of practical neural networks is a sufficiently difficult technical task.
JPA03-006679 discloses an invention for integrating the functions of one neuron onto one MOS (metal-oxide semiconductor structure) transistor so as to solve the above problem.
The above disclosed semiconductor device is a MOS semiconductor element called &ngr;MOS with a floating gate and a plural number of capacitance coupling input gates. Each of the input gates is equipped with an electrode having a proper area corresponding to its input weight. The input voltage provided to the input gate is multiplied with the proper weight determined by the electrode area. And the sum of the input voltages provided to the relating input gates corresponds to the voltage of the floating gate. When the sum exceeds a certain threshold level, a channel is formed under the floating gate electrode and an electric current runs through the channel as corresponding to the neuron's fire.
&ngr;MOS enables to decrease the transistor area in the semiconductor chip by tenth order as the number of the required bipolar transistors is decreased, and to realize low power consumption as MOS transistors are voltage-controlled devices. Thus the above approach is expected to obtain a real neuron computer.
According to the above technology in which the weights are determined by the capacitance coupling of the input gates to the floating gate, the number of the inputs and the weights to be applied to the inputs cannot change after building the device, because they are fixed by the structure of the &ngr;MOS. Therefore, the device is impossible to simulate flexible ability of the human brain relating information processings as recognition, association, learning, and so on. A method for solving the problem is proposed in which the device is furnished with multipliers for adjusting parameters and the adjusted parameters are applied to the input signals and then the input signals are provided to the &ngr;MOS. The device applied with the above method has much flexibility in functions though it needs more semiconductors per one synapse.
JPA6-139380 discloses a &ngr;MOS furnished with a self-learning facility. The disclosed device adjusts charges in the floating gate when the calculated value differs from the indicated value so as to adjust the weight of the synapse. This device can automatically adjust the weight function of the synapse which provides signals to the neuron, thus it can eliminate the need of an outstanding computer to calculate the weight of each synapse and may obtain an optimal control of the system through learning on the spot. Simulations have proven a high degree optimization in some logics to be executed.
JPA10-54079 discloses a &bgr;-driven threshold element (&bgr;DTE) invented by the inventor of this invention.
Any linear logics and some kind of logics can be represented in the threshold function below:
Y=
Sign(&Sgr;
j=0~n−1
&OHgr;jxj−T
)=Sign(&Sgr;
j&egr;!s
&OHgr;jxj−&Sgr;
j&egr;s
&OHgr;j!xj
)
where S is a subset of variables such that &Sgr;
j&egr;s
&OHgr;j=T, ! means negation, Xj is 0 or 1, and &OHgr; is an integer.
&OHgr; j is normalized with T to obtain &ohgr;j=&OHgr;j/T. Parallel connected p-channel MOS transistors whose input xj belongs to a certain subset S and its &bgr;-value is adjusted to the corresponding &ohgr;j are serially connected with parallel connected n-channel MOS transistors whose input does not belong to the subset S and its b-value is adjusted to the corresponding &ohgr;j. Then the output voltage Vout at the terminal connecting point is indicated as follows:
v
out=&Sgr;
j&egr;!s
&ohgr;jxj/&Sgr;
j&egr;s
&ohgr;j!xj
(1)
Therefore, a comparator inverter easily determines truth of the threshold function by comparing the output Vout with threshold value a, which is set as follows:
&agr;=&Sgr;
j&egr;!s
&ohgr;j/&Sgr;
j&egr;s
&ohgr;j!=&Sgr;&bgr;n/&Sgr;&bgr;p
(2).
The &bgr;DTE is constructed according to the above principle.
FIG. 13
is a circuit diagram indicating a principle of the &bgr;DTE.
Variables Xj belonging to the subset S in n number of Xj's described in the logics are provided to p-channel MOS transistors P
1
, P
2
, . . . Pk. Variables
Davis George B.
Monolith Company, Ltd.
Morrison & Foerster / LLP
LandOfFree
Artificial neuron on the base of B-driven threshold element does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Artificial neuron on the base of B-driven threshold element, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Artificial neuron on the base of B-driven threshold element will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2971789