Patent
1991-09-04
1994-03-15
Fleming, Michael R.
G06F 1518
Patent
active
052952282
ABSTRACT:
A learning machine has plural multiple-input single-output signal processing circuits connected in a hierarchical structure. The learning machine sets a threshold value, which is a evaluation standard for change in weight coefficients, high during the early part of the learning process and enables rough learning to progress without changing the weight coefficients for those multiple-input single-output signal processing circuits for which errors are sufficiently small. On the other hand, the learning machine gradually reduces the threshold value as learning progresses and advances learning by a non-linear optimization method (including a conjugate gradient method, a linear search method, or a combination of conjugate gradient and linear search methods) during the later part of the learning process, and thereby improves the learning speed.
REFERENCES:
patent: 4912652 (1990-03-01), Wood
patent: 5129038 (1992-07-01), Kohda et al.
Luenberger, D. G.; Introduction to Linear and Nonlinear Programming, Addison-Wesley Pub. Co., 1984, 243-244.
Rumelhart et al., Parallel Distributed Processing, vol. 1, Foundations, MIT Press, 1986, 318-362.
Dahl, E. D.; "Accelerated Learning Using the Generalized Delta Rule", IEEE First Intl. Con. on Neural Networks, Jun. 1987, II-523-II-530.
Parker, D. B.; "Optimal Algorithms for Adaptive Networks: Second Order Back Propagation, Second Order Direct Propagations, and Second Order Hebbian Learning" IEEE First Intl. Conf. on Neural Networks, Jun. 2, 1987, II593-II600.
Watrous , R. L.; "Learning Algorithms for Connectionist Networks; Applied Gradient Methods of Nonlinear Optimization", IEEE Intl. Conf. on Neural Networks, Jun. 1987, II-619-II-627.
Cater, J. P.; "Successfully Using Peak Learning Rates of 10 (and greater) in Back-propagation Networks with the Heuristic Learning Algorithm", IEEE First Intl. Conf. on Neural Networks, Jun. 1987, II-645-II-651.
Hush et al., "Improving the Learning Rate of Back-Propagation with the Gradient Reuse Algorithm", IEEE Intl. Conf. in Neural Networks, Jul. 1988, I-441-I-446.
Woods, D.; "Back and Counter Propagation Abberations", IEEE Intl. Conf. on Neural Networks, Jul. 1988, I-473-I-479.
Vogl et al., "Accelerating the Convergence of the Back-Propagation Method", Biological Cybernetics, 59, Spring 1988, 257-263.
Barnard et al., "A Comparison between Criterion Functions for Linear Classifiers, with an Application to Neural Nets", IEEE Trans. Syst., Man, and Cybernetics, Sep./Oct. 1989, 1030-1041.
Kramer et al., "Efficient Parallel Learning Algorithms for Neural Networks", in Advances in Neural Information Processsing Systems I, 1989, 40-48.
Huang et al., "Learning Algorithms for Perceptrons Using Back-Propagation with Selective Updates", IEEE Control Systems Magazine, Apr. 1990, 56-61.
Yan et al., "Weight Adjustment Rule of Neural Networks for Computing Discrete 2-D Gabor Transforms", IEEE Trans. ASSP, Sep. 1990, 1654-1656.
Battiti, R.; "First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method," Neural Computation 4, Mar. 1992, 1 41-166.
Yu, et al., "Training Algorithms for Back Propagation Neural Networks with Optimal Descent Factor," Electronics Letters, Sep. 1990, 1698-1700.
Koda Toshiyuki
Sakaue Shigeo
Shimeki Yasuharu
Yamamoto Hiroshi
Downs Robert W.
Fleming Michael R.
Matsushita Electric - Industrial Co., Ltd.
LandOfFree
Learning machine with a hierarchial structure without mutual con does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Learning machine with a hierarchial structure without mutual con, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Learning machine with a hierarchial structure without mutual con will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-1541696