Method and apparatus for determining the limit on learning machi

Details Method and apparatus for determining the limit on learning machi Method and apparatus for determining the limit on learning machi

1994-10-27

1997-11-04

Downs, Robert W.

395 23, G06F 1518

Patent

active

056849290

ABSTRACT:
A method and apparatus for determining the limit on learning machine accuracy imposed by the quality of data. A plurality of learning machines of increasing capacity are trained using training data and tested using test data, and the training error rates and test error rates are calculated. The asymptotic error rates of the learning machines are calculated and compared. When the change in asymptotic error rate falls below a certain rate, the asymptotic error rate estimates the limit on learning machine accuracy imposed by the data.

REFERENCES:
patent: 5033006 (1991-07-01), Ishizuka et al.
patent: 5052043 (1991-09-01), Gaborski
patent: 5095443 (1992-03-01), Watanabe
S. Bos, W. Kinzel, and M. Opper, Generalization Ability of Perceptrons with Continous Outputs, Physical Review E, vol. 47, No. 2, pp. 1384-1391, Feb. 1993.
W. P. Chiang, C. Cortes, L. D. Jackel, Y. LeCun, W. Lee, Accuracy Limits of Machine Learning: Predicting Communication Path Degradation or Failure, presented at Symposium on Intelligent Systems in Communications and Power, Puerto Rico, Feb. 21-23, 1994.
C. Cortes, Prediction of Generalization Ability in Learing Machines, PhD Thesis, University of Rochester, NY, 1993.
C. Cortes, L. D. Jackel, S. A. Solla, V. Vapnik, and J. S. Denker, Learing Curves: Asymptotic Values and Rate of Convergence, Advances in Neural Information Processing Systems, vol. 6, Morgan Kaufman, 1994.
T. L. Fine, Statistical Generalization and Learning, Technical Report EE577, Cornell University, Fall 1993.
K. Hornik, M. Stinchcombe, H. White, Multilayer Feedforward Networks are Universal Approximators, Neural Networks, vol. 2, pp. 359-366, 1989.
T. Kohonen, G. Barna, and R. Chrisley, Statistical Pattern Recognition with Neural Networks: Benchmarking Studies, Proc. IEEE Int. Conf on Neural Networks, ICCNN-88, vol. 1, p. I-61-I-68, 1988.
N. Murata, S. Yoshizawa, and S. Amari, Learning Curves, Model Selection, and Complexity of Neural Networks, Advances in Neural Information Processing Systems, vol. 5, pp. 607-614, Morgan Kaufman, 1992.
H. S. Seung, H. Sompolinsky, and N. Tishby, Statistical Mechanics of Learning From Examples, Physical Review A, vol. 45, No. 8 pp. 6056-6091, Apr. 15, 1992.
B. Moore, et al., "Characterizing the Error Function of a Neural Network," IEEE Symp. on the Frontiers of Massively Parallel Computation, pp. 49-57.
H. C. Leung and V. W. Zue, "Phonetic Classification Using Multi-Layer Perceptrons," 1990 Int'l. Conf. on Speech and Signal Processing, vol. 1, pp. 525-528.
K. A. Marko, et al., "Automotive Diagnostics Using Trainable Classifiers: Statistical Testing and Paradigm Selection," Int'l. Joint Conf. on Neural Networks 1990, vol. 1, pp. 33-38.
W. Gang and O.Y.J. Zheng, "On the Bound of the Approximation Capacity of Multi-Layer Neural Network," 1991 Int'l. Joint Conf. on Neural Networks, vol. 3, pp. 2299-2304.
T. S. Lin and J. Meador, "Classification-Accuracy Monitored Backpropagation," 1992 Int'l. Symp. on Circuits and Systems, vol. 3, pp. 1553-1556.
Ward Cheney and David Kincaid, Numerical Mathematics and Computing, Brooks Cole Publishing, p. 99, 1985.
John Hertz, Anders Krogh, Richard G. Palmer, Introduction to the Theory of Neural Computation, Addison-Wesley, pp., 225-226, 1991.
James L. McClelland and David E. Rumelhart, Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises, MIT Press, pp. 1-10.
Ester Levin, Naftali Tishby, and Sara A. Solla, A Statistical Approach to Learning and Generalization in layered Neural Networks, Proceedings of the IEEE, vol. 78, No. 10, pp. 1568-1574.
Naftali Tishby, Ester Levin, and Sara A. Solla, Consistent Inference of Probabilities in Layered Networks: Predictions and Generalizations, IEEE International Conference on Neural Networks, pp. 11-402 II-409, 1989.
Shun-ichi Amari, Learning Curves, Generalization Errors and Information Criteria, Artificial Neural Networks, 2, I. Aleksander and J. Taylor eds., Elsevier, pp. 305-311, 1992.
M. Stone, Cross-validatory Choice and Assessment of Statistical Predictions, J. Royal Stat. Soc., vol. 36, No. 1, pp. 111-147, 1974.
Shun-chi Amari and Noboru Murata, Statistical Theory of Learning Curves under Entropic Loss Criterion, Neural Computation, vol. 5, pp. 140-153, 1993.

Affiliated with

Also associated with

Rating

Method and apparatus for determining the limit on learning machi does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Method and apparatus for determining the limit on learning machi, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for determining the limit on learning machi will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-1840453