Data processing: artificial intelligence – Knowledge processing system – Creation or modification
Reexamination Certificate
2000-02-16
2003-05-13
Follansbee, John A. (Department: 2121)
Data processing: artificial intelligence
Knowledge processing system
Creation or modification
Reexamination Certificate
active
06564198
ABSTRACT:
TECHNICAL FIELD
The present invention relates generally to data processing systems and methods. More specifically, it relates to an artificial neural network-generated fuzzy expert system from which an accurate, compact, interpretable, and meaningful set of rules may be extracted.
BACKGROUND OF THE INVENTION
There are many approaches to data processing for developing rule sets for pattern recognition from provided data. Typical approaches utilize artificial neural networks (ANNs) or decision tree methods such as C5. The basic structure of an ANN comprises many layers of processing elements, which are referred to as neurons. The neurons in these many layers are interconnected by links that are assigned weight values during training. The weighted values are then interpreted to form rules to approximate the data. Data processing approaches such as the aforementioned find many uses in pattern recognition operations such as automotive occupant sensing and recognition, facial pattern recognition, gesture recognition, and object recognition, among others.
In some applications, such as automotive occupant sensing and recognition in particular, efficient operation is a key factor in success. In order to be practical, these methods must satisfy four critical constraints. The first constraint is that the methods must be extremely accuarate so that they can correctly handle the large number of finely differentiated possible geometric configurations for a vehicle occupant. The second constraint is that the method must have a fast response time. This is required to provide sufficient time for deployment of mechanical hardware, such as airbag systems, during collisions/accidents. The third constraint is that the method allow for the rationale for its actions under various situations to be understood and interpreted by humans. Human understanding and interpretation of occupant sensing and recognition methods is very important for product development, support, and analysis purposes. The last constraint is that the method must be inexpensive to implement in hardware. This is necessary to allow feasible implementation in an automobile and to provide an economic competitive advantage in the marketplace.
ANNs and C5 decision tree networks have previously been applied to pattern recognition operations. With regard to ANNs, the main disadvantage is the inability to explain learned knowledge from ANNs in a manner that can be easily understood by humans. As stated before, the ability to generate explainable rules is important for product development, support, and analysis purposes. The C5 decision tree network satisfies the aforementioned constraints to a degree. However, it is still desirable to provide a greater degree of accuracy and a more compact rule set.
ANNs, while capable of providing compact, highly accurate rule sets, have been criticized as being “black boxes” because their behavior has historically been unexplainable. In the article entitled “Are Artificial Neural Networks Black Boxes?”, IEEE Transactions on Neural Networks, Vol. 8, No. 5, September 1997, incorporated herein by reference, Benitez, Castro, and Requena attempted to solve this problem by developing a new fuzzy-logic operator termed the interactive-or, or I-OR, operator. The interactive-or operator may be used to derive fuzzy rules from a fully trained ANN. While the method developed by Benitez et al. is able to extract fuzzy rules, the rules are not easily interpretable by humans because there is no assurance that the values of the input features, as reflected in the antecedents, of each fuzzy rule will fall within the allowable range of each input feature. In fact, although a particular antecedent may be unimportant to a particular rule, in many cases, all of the antecedents may exceed the range used to train the neural network. Finally, the output values, or consequents, are expressed as numeric values, further reducing the interpretability of the extracted rules.
A simplified example of a three-layered ANN, comprising an input layer
100
, a hidden layer
102
, and an output layer
104
is shown in FIG.
1
. As shown, the input layer
100
includes two input nodes, X
1
106
and X
2
108
, which provide data to the network. The hidden layer
102
includes two hidden layer nodes, H
1
110
and H
2
112
. The hidden layer nodes H
1
110
and H
2
112
each correspond to a unique fuzzy rule where, in the general case, the total number of hidden layer nodes H
j
corresponds to the total number of rules j in the system. As shown in the diagram, the hidden layer nodes, H
1
110
and H
2
112
also provide the output variables Y
1
and Y
2
for the generation of the rule base. In the example of
FIG. 1
, therefore, there are two rules in the rule base because there are two hidden layer nodes, H
1
110
and H
2
112
. Specifically, are as many rules j as there are nodes in the hidden layer
102
. According to the work of Benitez et al., the rules j for the hidden layer nodes, H
1
110
and H
2
112
may be formulated as:
Rule 1: IF {
X
1
is
A
1
} I-OR {
X
2
is
B
1
} THEN {
Y
1
is
C
1}
and
Rule 2: IF {
X
1
is
A
2
} I-OR {
X
2
is
B
2
} THEN {
Y
2
is
C
2
},
where {A
i
, B
i
, C
i
} represent the fuzzy sets that describe the input variables {X
1
, X
2
} and the output variables { Y
1
, Y
2
} for each rule. The terms in brackets { } to the left of THEN correspond to the “antecedents” for each rule. The terms in brackets { } to the right of THEN correspond to the “consequents” for each rule. In general, there are as many antecedents as inputs X
i
in the input layer
100
with i−1 interactive or terms between them. Thus, given two inputs X
1
and X
2
, two antecedents would be combined into rules as shown above for the two input case with one I-OR term between them.
The rules above appear similar to rules found in traditional fuzzy logic systems, except for the presence of the I-OR terms. Two important features in the rule formulation above add clarity to the similarity between traditional fuzzy logic systems and the I-OR function. The first feature relates to the explainability of the rules. In a traditional fuzzy logic system, fuzzy sets are expressed in terms of linguistic labels such as SMALL, TALL, etc., and not with numeric values. Thus, they are more readily understandable and interpretable by humans. The analogous interpretation for the fuzzy set of each antecedent (e.g. (A
1
and B
1
) in the Rule 1, above) for a given rule j was derived from the neural network described in the work of Benitez et al. to be of the form “X
i
is greater/lower than approximately (2.2−T
j
/2)/W
ij
”. The value of 2.2 was obtained by inverting the unipolar sigmoidal activation function,
f
(
x
)
=
1
1
+
ⅇ
-
W
ij
X
i
+
T
j
at an activation value (chosen at 0.9). The unipolar sigmoidal activation function serves as the membership function for each fuzzy set, similar to the trapezoidal/triangular membership functions found in fuzzy logic systems. It is important to note that the sigmoidal function may take any applicable form, and may be unipolar or bipolar as desired. The term W
ij
corresponds to the weight between the input node X
i
and the hidden layer node H
j
used in the generation of the rule j, and the appearance of the “greater” or “lesser” term depends on whether W
ij
is positive or negative, respectively. The threshold T
j
for a given rule rule j is equally partitioned between all of its antecedents. The consequents C
j
are directly set to the weight values Z
j
(i.e., no linguistic label). The second feature concerns the manner in which antecedents of a rule are combined to form a fuzzy rule. In fuzzy logic-based systems, the antecedents are combined using the AND/OR operators. However, as discussed in the article “The Representation of Fuzzy Relational Production Rules”, Applied Intelligence, Vol. 1, Issue 1, p. 35-42 1991 by R. R. Yager, it has been proven that AN
Narayan Srinivasa
Owechko Yuri
Follansbee John A.
Holmes Michael B.
HRL Laboratories LLC
Tope-McKay & Associates
LandOfFree
Fuzzy expert system for interpretable rule extraction from... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Fuzzy expert system for interpretable rule extraction from..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Fuzzy expert system for interpretable rule extraction from... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3047194