Optimum cessation of training in neural networks

Details Optimum cessation of training in neural networks Optimum cessation of training in neural networks

1997-11-19

2000-09-12

Hafiz, Tariq R.

Data processing: artificial intelligence

Neural network

Learning method

706 26, G06N 308

Patent

active

061191124

ABSTRACT:
A system and method for training a neural network that ceases training at or near the optimally trained point is presented. A neural network having an input layer, a hidden layer, and an output layer with each layer having one or more nodes is presented. Each node in the input layer is connected to each node in the hidden layer and each node in the hidden layer is connected to each node in the output layer. Each connection between nodes has an associated weight. All nodes in the input layer are connected to a different historical datum from the set of historical data. The neural network being operative by outputting a prediction or classification, the output of the output layer nodes, when presented with input data. The weights associated with the connections of the neural network are first adjusted by a training device. The training device then iteratively applies a training set to the neural network, the training set consisting of historical data. After each iteration the weights associated with the connections are adjusted according to the difference between the prediction or classification produced by the neural network given the training data and the known prediction or classification of the historical data. Additionally, after each iteration, a test set, consisting of different historical data from that in the training set, is presented to the neural network. The training device then determines the difference between the known result from the test set and the result from the presentation of the test set to the neural network. This difference, herein referred to as the variance, is then recorded along with the weights in the neural network. The variance is monitored at each iteration to determine if it is monotonically, within a given margin of error, decreasing. That is the prediction or classification resulting from the test set being presented to the neural network is getting successively closer to matching the known result from the test set. When the variance hits the inflection point where it begins to increase, training is ceased. At this point the neural network is no longer learning the pattern underlying the input data, but is instead over fitting the input data.

REFERENCES:
patent: 5461699 (1995-10-01), Arbabi et al.
"Practical Neural Network Recipes in C++"; by Timothy Masters; edition (Apr. 1993); Academic Pr; ISBN: 0124790402.
NeuroShell.RTM. 2; Ward Systems Group, Inc.; www.wardsystems.com, 1993.

Affiliated with

Also associated with

Rating

Optimum cessation of training in neural networks does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Optimum cessation of training in neural networks, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Optimum cessation of training in neural networks will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-105081