Neuron, hierarchical neural network using the neuron, and...

Details Neuron, hierarchical neural network using the neuron, and... Neuron, hierarchical neural network using the neuron, and...

2000-05-24

2002-12-17

Black, Thomas (Department: 2121)

Data processing: artificial intelligence

Neural network

Structure

C332S106000

Reexamination Certificate

active

06496815

ABSTRACT:

CROSS REFERENCE TO RELATED APPLICATION
This application is based upon Japanese Patent Application Nos. Hei. 11-148067 filed on May 27, 1999, and Hei. 11-328312 filed on Nov. 18, 1999, the contents of which are incorporated herein by reference.
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates to signal processing apparatuses, and particular to a signal processing apparatus such as a neural computer applied in character and graphic recognition, associated storage, multi-input/output non-linear mapping and the like.
2. Related Art
Hitherto, there has been known a neural network (neural cell circuit network) modeled after information processing carried out in a living body. In the neural network, a neuron is a unit of function and information processing is carried out by disposing a plurality of neurons in network. Such neural network is suitable for information processing of character and graphic recognition, associated storage, multi-input/output non-linear mapping and the like which can be hardly achieved by the conventional Neumann type computers.
Next, the neural network will be explained to facilitate understanding of the present invention.
At first, the schematic structure of the neural network will be explained.
The neural network is configured by disposing neurons in network as described above as shown in
FIG. 15
for example.
The neural network shown in
FIG. 15
is called as a three-layered hierarchical neural network and comprises an input layer, an intermediate layer (hidden layer) and an output layer.
It is noted that a signal is inputted from the input layer and is outputted from the output layer as it propagates sequentially through the intermediate layer and the output layer. The input layer only propagates the input signal to the intermediate layer and carries out no arithmetic operation like the intermediate layer and the output layer as is known in the technological field. Therefore, the functional unit composing the intermediate and output layers is referred to as a neuron. The intermediate and output layers contain at least one neuron, respectively.
The input layer is coupled with the respective neurons of the intermediate layer and the respective neurons of the intermediate layer are coupled with the respective neurons of the output layer as shown in FIG.
15
. Then, the signal inputted to the input layer of the neural network is propagated to the intermediate layer to undergo predetermined arithmetic operations within the neurons contained in the intermediate layer as described later. Its output value is propagated further to the output layer. Similar arithmetic operations are carried also in the neurons contained in the output layer and its output value becomes the final output of the network.
This series of operations is the information processing of the neural network called a sequential propagation (forward processing) and allows input/output to be realized arbitrarily when a sufficient number of neurons are contained in the intermediate layer.
It is noted that although the neural network shown in
FIG. 15
is a three-layered structure network having one intermediate layer, there has been proposed a network having two or more intermediate layers.
The neuron which is the structural unit of the neural network will be explained next.
FIG. 16
is a diagrammatic view of the j-th neuron denoted by a symbol j in FIG.
15
. The neuron is composed of an input section for inputting input values from the outside, a computing section for computing those input values and an output section for outputting the result of computation.
When each input value from the outside is expressed as x
i
(i=1, 2, 3, . . . , n), the computing section multiplies a corresponding link weight w
ji
(i=1, 2, 3, . . . , n) with each input value x
i
and calculates their sum y
j
as shown by the following expression (1):
y
j
=&Sgr;w
ji
x
i
  (1)
It is noted that the symbol &Sgr; is a symbol of sum of i. The link weight w
ji
indicates the strength of coupling between the neurons. The w
ji
indicates the link weight between the j-th neuron and the i-th neuron.
The computing section executes a non-linear computation f to the sum y
j
found as described above to output an output value z
j
as expressed by the following expression (2):
z
j
=f
(
y
i
)  (2)
A Sigmoid function is often used as the non-linear function f because it expresses a differential value f′ of the non-linear function f which is required in realizing a learning function described next by using the non-linear function f itself like f′=f·(1−f) and allows an amount of computation to be reduced. A step function is used as the non-linear function f in some cases. However, the non-linear function is not limited to those functions and may be a monotonous increment function having a saturating characteristic.
The neural network having such neurons as the structural unit is characterized in that it has the learning function. This learning function will be explained next.
The learning in the neural network is realized by updating the link weight of each neuron described above. That is, it enables to obtain a desired output signal z from the output layer when the value of link weight w is updated sequentially to an adequate value and a set of input signals (input pattern) p is given to the input layer.
In executing learning, a desired output signal t corresponding to the input signal p is given together with the input signal p. This output signal t is referred to as a teacher signal. A learning method called back propagation (BP) algorithm is used in the hierarchical neural network shown in FIG.
15
.
The back propagation (BP) will be explained concretely.
When a certain input signal p is given, a square error of an output value z
k
of the neuron of the k-th output layer and a teacher signal value t
k
is defined by the following expression:
E
k
=(
t
k
−z
k
)
2
/2  (3)
In the learning, the degree of all link weights is updated so as to reduce this square error E
k
.
When the link weight W
kj
of the j-th neuron in the intermediate layer and the k-th neuron in the output layer is updated, the square error E
k
varies as follows:
∂
E
k
/∂w
kj
=−(
t
k
−z
k
)·
f
′(
y
k
)·
z
j
  (4)
Here, z
j
is an output of the j-th neuron of the intermediate layer and f′ is differential of non-linear function of the neuron. Y
k
is an input sum expressed by the expression (1) described above about the k-th neuron of the output layer.
When the link weight w
ji
of the i-th neuron in the input layer and the j-th neuron in the intermediate layer is updated, the square error E
k
varies as follows:
 ∂
E
k
/∂w
ji
=−{&Sgr;(
t
k
−z
k
)·
f
′(
y
k
)·
w
kj
}·f
′(
y
j
)·
z
i
  (5)
Here, the symbol &Sgr; is a symbol of sum about k. z
i
is an output of the i-th neuron of the input layer and y
j
is an input sum expressed by the expression (1) described above about the j-th neuron of the intermediate layer.
Accordingly, the amounts of update &Dgr;w
kj
and &Dgr;w
ji
of the link weights for reducing the square error may be expressed by the following expressions:
&Dgr;
w
kj
=w
kj
(
t
+1)−
w
kj
(
t
)=−&eegr;·∂
E
k
/∂w
kj
  (6)
&Dgr;
w
ji
=w
ji
(
t
+1)−
w
ji
(
t
)=−&eegr;·∂
E
k
/∂w
ji
  (7)
It is noted that t indicates time here. The &eegr; is a positive number called an update weight and is normally determined experimentally in a range of 0.1 to 0.5.
While the outline of the neural network has been explained above in detail, how to realize the function of the neuron described above is questioned in configuring the neural network.
Hitherto, the method of realizing the function of neuron by processing in software by using the Neumann type computer has been used often. However, the original parallel information processing is not car

Affiliated with

Also associated with

Rating

Neuron, hierarchical neural network using the neuron, and... does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Neuron, hierarchical neural network using the neuron, and..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Neuron, hierarchical neural network using the neuron, and... will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-2997724