Method and apparatus for adaptive learning in neural networks

Details Method and apparatus for adaptive learning in neural networks Method and apparatus for adaptive learning in neural networks

1993-04-22

1995-03-14

Downs, Robert W.

G06F 1518

Patent

active

053983025

ABSTRACT:
A method enabling a neural network, having weights organized into a weight vector, to learn, comprising the steps of: (a) assigning a first memory location for storing a first learning rate, a second memory location for storing a momentum factor, a memory block for storing the weight vector, and a third memory location for storing a second learning rate; (b) initializing the learning rate, momentum factor, and weight vector; (c) storing the first learning rate, the momentum factor, and the weight vector into their respective memory locations; (d) saving the first learning rate in the second learning rate by storing it into the third memory location; (e) using a search technique to adjust the first learning rate to adjust the weight vector, and updating the first memory location and the memory block; (f) adapting the momentum factor using the first learning rate and the second learning rate; and repeating steps (c) through (f) until a predetermined convergence criterion has been met.

REFERENCES:
patent: 4912651 (1990-03-01), Wood et al.
Hash, D. R., Salas, J. M., "Improving the Learning Rate of Back-Propagation with the Gradient Reuse Algorithm.", from IEEE Int. Conf. on Neural Networks, Jul. 1988, pp. I-441-I-446.
Kollias, S. et al., "Adaptive Training of Multilayer Neural Networks Using a Least Squares Estimation Technique", from IEEE Intl. Conf. on Neural Networks, Jul. 1988, pp. I-383-I-390.
Lippmann, R. P., "An Introduction to Computing with Neural Nets.", IEEE ASSP Mag., Apr. 1987, pp. 4-22.
McClelland, J. L., Rumelhart, D. E, "Explorations in Parallel Distributed Processing", The Mit Press, 1988, pp. 121-152.
Fanelli, R. et al, "Neural Network Modelling of Eye Compensation During Off-Vertical-Axis Rotation", Neural Networks, vol. 3, 1990, pp. 265-276.
Vogl, et al., "Accelerationg the Convergence of the Back-Propagation Method," Biological Cybernetics, 59, pp. 257-263, (1988).
Jacobs, R. A., "Increased Rates of Convergence Through Learning Rate Adaptation," Neural Networks, vol. 1, (1988), pp. 295-307.
Dahl, E. D., "Accelerated Learning using the Generalized Delto Rule," Proceed. 1st Intl. Conf. on Neural Networks (1987), II-523-II-530.
Strang, G., Introduction to Applied Mathematics, Wellesley-Cambridg Press, (1986), pp. 367-424.
Chong et al., "Inplementation of Neural Networks for Speech Recognition on a Transputer Array"0 Cambridge University Dept. of Engineering, Mar. 1988.

Affiliated with

Also associated with

Rating

Method and apparatus for adaptive learning in neural networks does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Method and apparatus for adaptive learning in neural networks, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for adaptive learning in neural networks will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-718063