Data processing: artificial intelligence – Fuzzy logic hardware – Fuzzy neural network
Reexamination Certificate
1999-03-18
2002-07-23
Black, Thomas (Department: 2122)
Data processing: artificial intelligence
Fuzzy logic hardware
Fuzzy neural network
C706S021000, C706S037000, C706S041000
Reexamination Certificate
active
06424956
ABSTRACT:
FIELD AND BACKGROUND OF THE INVENTION
The present invention relates-in general to artificial intelligence systems and in particular to a new and useful device which combines artificial neural network (“ANN”) learning techniques with fuzzy logic techniques.
Both neural network learning techniques and fuzzy logic techniques are known. In fact, prior combinations of the two techniques are known as well, as for example U.S. Pat. No. 5,179,624 issued Jan. 12, 1993 to Amano (“Speech recognition apparatus using neural network and fuzzy logic”), which is incorporated herein by reference.
Both techniques attempt to replicate or improve upon a human expert's ability to provide a response to a set of inputs. ANNs extract knowledge from empirical databases used as training sets and fuzzy logic usually extracts rules from human experts.
In very brief summary, neural network techniques are based on observation of what an expert does in response to a set of inputs, while fuzzy logic techniques are based on eliciting what an expert says he will do in response to a set of inputs. Many authors, including Applicant, have recognized the potential value of combining the capabilities of the two techniques.
Applicant is the author of Chapters 3, 10 and 13 of D. White & D. Sofge,
Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches
, Van Nostrand, 1992, (“HIC”), which was published no earlier than Sep. 1, 1992 and which contains disclosure of a number of novel inventions which will be summarized and claimed herein. The entirety of those chapters are incorporated herein by reference.
The invention described and claimed herein comprises an Elastic Fuzzy Logic (“ELF”) System in which classical neural network learning techniques are combined with fuzzy logic techniques in order to accomplish artificial intelligence tasks such as pattern recognition, expert cloning and trajectory control. The ELF system may be implemented in a computer provided with multiplier means and storage means for storing a vector of weights to be used as multiplier factors in an apparatus for fuzzy control. The invention further comprises novel techniques and apparatus for adapting ELF Systems and other nonlinear differentiable systems and a novel gradient-based technique and apparatus for matching both predicted outputs and derivatives to actual outputs and derivatives of a system.
NEURAL NETWORKS
Artificial Neural Networks (“ANNs”) are well known, and are described in general in U.S. Pat. No. 4,912,654 issued Mar. 27, 1990 to Wood (“Neural networks learning method”) and in U.S. Pat. No. 5,222,194 issued Jun. 22, 1993 to Nishimura (“Neural network with modification of neuron weights and reaction coefficient”), both of which are incorporated herein by reference.
ANNs typically are used to learn static mappings from an “input vector,” X, to a “target vector,” Y. The first task is to provide a training set—a database—that consists of sensor inputs (X) and desired actions (y or u). The training set may, for example, be built by asking a human expert to perform the desired task and recording what the human sees (X) and what the human does (y). Once this training set is available, there are many neural network designs and learning rules (like basic backpropagation) that can learn the mapping from X to y. Given a training set made up of pairs of X and y, the network can “learn” the mapping by adjusting its weights so as to perform well on the training set. This kind of learning is called “supervised learning” or “supervised control”. Advanced practitioners of supervised control no longer think of supervised control as a simple matter of mapping X(t), at time t, onto y(t). Instead, they use past information as well to predict y(t).
Broadly speaking, neural networks have been used in control applications:
1. As subsystems used for pattern recognition, diagnostics, sensor fusion, dynamic system identification, and the like;
2. As “clones” which learn to imitate human or artificial experts by copying what the expert does;
3. As “tracking” systems, which learn strategies of action which try to make an external environment adhere to a pre-selected reference model.
(4) As systems for maximizing or minimizing a performance measure over time. For true dynamic optimization problems, there are two methods of real use: (1) the backpropagation of utility (which may be combined with random search methods); (2) adaptive critics or approximate dynamic programming. The backpropagation of utility is easier and more exact, but it is less powerful and less able to handle noise. Basic backpropagation is simply a unique implementation of least squares estimation. In basic backpropagation, one uses a special, efficient technique to calculate the derivatives of square error with respect to all the weights or parameters in an ANN; then, one adjusts the weights in proportion to these derivatives, iteratively, until the derivatives go to zero. The components of X and Y may be 1's and 0's, or they may be continuous variables in some finite range. There are three versions of backpropagating utility: (1) backpropagating utility by backpropagation through time, which is highly efficient even for large problems but is not a true real-time learning method; (2) the forward perturbation method, which runs in real time but requires too much computing power as the size of the system grows; (3) the truncation method, which fails to account for essential dynamics, and is useful only in those simple tracking applications where the resulting loss in performance is acceptable. D. White & D. Sofge,
Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches
, Van Nostrand, 1992, (“HIC”) describes these methods in detail and gives pseudocode for “main programs” which can be used to adapt any network or system for which the dual subroutine is known. The pseudocode for the ELF and F_ELF subroutines provided below may be incorporated into those main programs (though the F_X derivatives need to be added in some cases).
Backpropagation cannot be used to adapt the weights in the more conventional, Boolean logic network. However, since fuzzy logic rules are differentiable, fuzzy logic and backpropagation are more compatible. Strictly speaking, it is not necessary that a function be everywhere differentiable to use backpropagation; it is enough that it be continuous and be differentiable almost everywhere. Still, one might expect better results from using backpropagation with modified fuzzy logics, which avoid rigid sharp corners like those of the minimization operator.
One widely used neural network (a multi-layer perceptron) includes a plurality of processing elements called neural units arranged in layers. Interconnections are made between units of successive layers. A network has an input layer, an output layer, and one or more “hidden” layers in between. The hidden layer is necessary to allow solutions of nonlinear problems. Each unit is capable of generating an output signal which is determined by the weighted sum of input signals it receives and a threshold specific to that unit. A unit is provided with inputs (either from outside the network or from other units) and uses these to compute a linear or non-linear output. The unit's output goes either to other units in subsequent layers or to outside the network. The input signals to each unit are weighted either positively or negatively, by factors derived in a learning process.
When the weight and threshold factors have been set to correct levels, a complex stimulus pattern at the input layer successively propagates between hidden layers, to result in an output pattern. The network is “taught” by feeding it a succession of input patterns and corresponding expected output patterns; the network “learns” by measuring the difference (at each output unit) between the expected output pattern and the pattern that it just produced. Having done this, the internal weights and thresholds are modified by a learning algorithm to provide an output pattern which more closely approximates the exp
Black Thomas
Starkes Wilbert L.
LandOfFree
Stochastic encoder/decoder/predictor does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Stochastic encoder/decoder/predictor, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Stochastic encoder/decoder/predictor will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2891138