Data processing: artificial intelligence – Fuzzy logic hardware – Fuzzy neural network
Reexamination Certificate
1999-02-22
2002-09-24
Black, Thomas (Department: 2122)
Data processing: artificial intelligence
Fuzzy logic hardware
Fuzzy neural network
C706S004000, C706S006000, C706S047000
Reexamination Certificate
active
06456989
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates to a data processing ing system in a hierarchical i.e. layered network configuration for flexibly processing data in a comprehensible and executable form, and, more specifically, to a neuro-fuzzy-integrated data processing system in a neuro-fuzzy-integrated hierarchical network configuration for establishing high speed, high precision data processing capabilities.
BACKGROUND OF THE INVENTION
In a conventional serial processing computer (Neiman computer), it is very difficult to adjust data processing capabilities with the change of usage and environment. Accordingly, more flexible data processing units have been demanded and developed to process data using a parallel and distributed method in a new hierarchical network configuration specifically in the technical field of pattern recognition, applicable filter, etc. In this hierarchical network data processing unit, programs are not written explicitly, but an output signal (output pattern) provided by a network structure in response to an input signal (input pattern) presented for learning determines a weight value of an internal connection of a hierarchical network structure so that the output signal corresponds to a teaching signal (teacher pattern) according to a predetermined learning algorithm. When the weight value is determined by the above described learning process, a “flexible” data processing function can be realized such that the hierarchical network structure outputs a probable output signal even though an unexpected input signal is inputted.
In the data processing unit in this hierarchical network configuration, there is an advantage in that a weight value of an internal connection can be automatically determined only if a learning signal is generated. However, there is also a disadvantage in that the content of a data process which is dependent on a weight value is not comprehensible. A successful solution would allow this data processing unit in a network structure to be put into practical use. To practically control this data processing unit in a hierarchical network configuration, an effective means must be presented for high speed, high precision data processing.
In a data processing unit comprising a hierarchical network configuration, a hierarchical network comprises one kind of node called a “basic unit” and an internal connection having a weight value corresponding to an internal state value.
FIG. 1
shows a basic configuration of a basic unit
1
. The basic unit
1
is a multi-input/output(i/o) system. That is, it comprises a multiplier
2
for multiplying a plurality of inputs by respective weight values of an internal connection, an accumulator
3
for adding all products of the above described multiplications performed by the multiplier, and a function converter
4
for applying function conversion such as a non-linear threshold process to the resultant accumulation value so that a final output can be obtained.
Assuming that a layer h is a preprocess layer and a layer i is a post-process layer, the accumulator
3
of the i-th basic unit
1
in the layer i executes the following operation using expression (1), and the function converter
4
executes a threshold operation using expression (2).
x
pi
=
∑
h
y
ph
W
ih
(
1
)
y
pi
=1/(1+exp(−
x
pi
+&thgr;
i
)) (2)
where
h: a unit number in the layer h
p: a pattern number of an input signal
&thgr;
1
: a threshold of the i-th unit in the layer i
W
ih
: a weight value of an internal connection between the layers h and i
Y
ph
: output from the h-th unit in the layer h in response to an input signal in the p-th pattern.
The data processing unit in a hierarchical network configuration configures its hierarchical network such that a plurality of the above described basic units
1
are hierarchically connected as shown in
FIG. 2
(with input signal values distributed and outputted as an input layer 1′) thus performing a parallel data process by converting input signals into corresponding output signals.
The data processing unit in a hierarchical network configuration requires obtaining, by a learning process, a weight value of a hierarchical network structure which determines the data conversion. Specifically, a back propagation method attracts special attention for its practicality as an algorithm of the learning process. In the back propagation method, a learning process is performed by automatically adjusting a weight value W
ih
and a threshold &thgr;
i
through feedback of an error. As indicated by expressions (1) and (2), the weight value W
ih
and the threshold &thgr;
i
must be adjusted simultaneously, but the adjustment is very difficult because these values must be carefully balanced. Therefore, the threshold &thgr;
i
is included in the weight value W
ih
by providing a unit for constantly outputting “1” in the layer h on the input side and assigning the threshold &thgr;
i
as a weight value to the output, therefore allowing it to be processed as a part of the weight value. Thus, expressions (1) and (2) are represented as follows:
x
pi
=
∑
h
y
ph
W
ih
(
3
)
y
pi
=1/(1+exp(−
x
pi
)) (4)
In the back propagation method, as shown in the three-layer structure comprising a layer h, a layer i and a layer j in
FIG. 2
, the difference (d
pj
−y
pj
) between an output signal y
pj
and a teaching signal d
pj
is calculated when the output signal y
pj
outputted from the output layer in response to an input signal presented for learning and a teaching signal d
pj
for matching the output signal y
pj
are given. Then, the following operation is performed:
&agr;
pj
=y
pj
(1−
y
pj
)(
d
pj
−y
pj
) (5)
followed by:
Δ
W
ji
(
t
)
=
ϵ
∑
p
α
pj
y
pi
+
ζΔ
W
ji
(
t
-
1
)
(
6
)
Thus, an updated weight value &Dgr;W
ji
(t) between the layers i and j is calculated, where t indicates the count of learnings.
Then, using the resultant &agr;
pj
, the following operation is performed:
β
pi
=
y
pi
(
1
-
y
pi
)
∑
j
α
pj
W
ji
(
t
-
1
)
(
7
)
followed by:
Δ
W
ih
(
t
)
=
ϵ
∑
p
β
pj
y
ph
+
ζΔ
W
ih
(
t
-
1
)
(
8
)
Thus, an updated weight value &Dgr;W
ih
(t) between the layers h and i is calculated.
Then, weight values are determined for the following update cycles according to the updated value calculated as described above:
W
ji
(
t
)
=
W
ji
(
t
-
1
)
+
Δ
W
ji
(
t
)
W
ih
(
t
)
=
W
ih
(
t
-
1
)
+
Δ
W
ih
(
t
)
}
(8a)
By repeating the procedure above, learning is completed when the weight values W
ji
and W
ih
are obtained where an output signal y
pj
outputted from the output layer in response to an input signal presented for learning corresponds to a teaching signal d
pj
, a target of the output signal y
pj
.
When the hierarchical network has a four-layer configuration comprising layers g, h, i, and j, the following operation is performed:
γ
ph
=
y
ph
(
1
-
y
ph
)
∑
i
β
pi
W
ih
(
t
-
1
)
(
9
)
followed by:
Δ
W
hg
(
t
)
=
ϵ
∑
p
γ
ph
y
pg
+
ζΔ
W
hg
(
t
-
1
)
(
10
)
Thus, the updated amount &Dgr;W
hg
(t) of a weight value between the layers g and h can be calculated. That is, an updated amount &Dgr;W of a weight value between the preceding layers can be determined from the value obtained at the last step at the output side and the network output data.
If the function converter
4
of the basic unit
1
performs linear conversion, the expression (5) above is represented as follows:
&agr;
pj
=(
d
pj
−y
pj
) (11)
the expression (7) above is re
Asakawa Kazuo
Kawamura Akira
Masuoka Ryushuke
Matsuoka Shigenori
Okada Hiroyuki
Black Thomas
Booker Kelvin
Fujitsu Limited
Staas & Halsey , LLP
LandOfFree
Neuro-fuzzy-integrated data processing system does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Neuro-fuzzy-integrated data processing system, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Neuro-fuzzy-integrated data processing system will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2819033