Training a learning system with arbitrary cost functions

Details Training a learning system with arbitrary cost functions Training a learning system with arbitrary cost functions

2005-12-16

2008-12-30

Vincent, David R (Department: 2129)

Data processing: artificial intelligence

Neural network

Learning task

C382S159000, C382S181000

Reexamination Certificate

active

07472096

ABSTRACT:
The subject disclosure pertains to systems and methods for training machine learning systems. Many cost functions are not smooth or differentiable and cannot easily be used during training of a machine learning system. The machine learning system can include a set of estimated gradients based at least in part upon the ranked or sorted results generated by the learning system. The estimated gradients can be selected to reflect the requirements of a cost function and utilized instead of the cost function to determine or modify the parameters of the learning system during training of the learning system.

REFERENCES:
patent: 5625751 (1997-04-01), Brandwajn et al.
patent: 6260013 (2001-07-01), Sejnoha
patent: 6526440 (2003-02-01), Bharat
patent: 7281002 (2007-10-01), Farrell
patent: 2003/0236662 (2003-12-01), Goodman
patent: 2005/0049990 (2005-03-01), Milenova et al.
patent: 2005/0144158 (2005-06-01), Capper et al.
Jarvelin et al, “Cumulated Gain-Based Evaluation of IR Techniques”, 2002.
Storn et al., “Differential Evolution—A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces”, 1996.
Xia et al., “A One-Layer Recurrent Neural Network for Support Vector Machine Learning”, 2004.
Cohen et al, “Volume Seedlings”, 1992.
Storn, “On the Usage of Differential Evolution for Function Optimization”, 2002.
Storn, et al., “Differential Evolution—A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces”, 1996.
Xia, et al., “A One-Layer Recurrent Neural Network for Support Vector Machine Learning”, 2004.
Cohen, et al., “Volume Seedlings”, 1992.
Jarvelin, et al., Cumulated Gain-Based Evaluation of IR Techniques, 2002.
International Search Report and Written Opinion dated Mar. 6, 2008 for PCT Application Serial No. PCT/US06/26266, 11 Pages.
Joachims. “Optimizing Search Engines using Clickthrough Data” ACM SIGKDD 02, Edmonton, Alberta, Canada. pp. 133-142. Last accessed Jun. 26, 2008, 10 pages.
OA dated Jun. 26, 2008 for U.S. Appl. No. 11/378,086, 27 pages.
G. S. Kimeldorf, et al., “Some results on Tchebycheffian Spline Functions” J. Mathematical Analysis and Applications, 1971, vol. 33, pp. 82-95.
C. Burges, et al, “Learning to Rank Using Gradient Descent”, Proceedings of the 22nd international conference on Machine learning, ACM International Conference Proceeding Series; 2005, pp. 89-96, vol. 119, Bonn, Germany.
C. Burges, “Ranking as Learning Structured Outputs”, in Proceedings of the NIPS 2005 Workshop on Learning to Rank, Dec. 2005, 4 pages.
B. Scholkopf, et al., “Learning with Kernels”, MIT Press, 2002, pt. 1, 100 pages (front cover-80).
B. Scholkopf, et al., “Learning with Kernels”, MIT Press, 2002, pt. 2, 100 pages (81-180).
B. Scholkopf, et al., “Learning with Kernels”, MIT Press, 2002, pt. 3, 100 pages (181-280).
B. Scholkopf, et al., “Learning with Kernels”, MIT Press, 2002, pt. 4, 100 pages (281-380).
B. Scholkopf, et al., “Learning with Kernels”, MIT Press, 2002, pt. 5, 100 pages (381-480).
B. Scholkopf, et al., “Learning with Kernels”, MIT Press, 2002, pt. 6, 100 pages (481-580).
B. Scholkopf, et al., “Learning with Kernels”, MIT Press, 2002, pt. 7, 49 pages (581-back cover).
Freund, et al. “An Efficient Boosting Algorithm for Combining Preferences” (1999) 9 pages.
Bromley, et al. “Signature Verification Using ‘Siamese’ Time Delay Nural Network.” (1993) Advances in Pattern Recognition Systems Using Neural Network Technologies, World Scientific, pp. 25-44.
Burges, C. “Simplified Support Vector Decision Rules” (1996) International Conference on Machine Learning, pp. 71-77.
Dekel, et al. “Log-linear Models for Label-ranking” (2004) NIPS, 8 pages.
Harrington, E. “Online ranking/collaborative filtering Using Perceptron Algorithm” (2003) ICNL, 8 pages.
Hastie, et al. “Classification by Pairwise Coupling” (1998) NIPS, pp. 451-471.
Jarvelin, et al. “IR Evaluation Methods for Retrieving Highly Relevant Documents” (2000) Proceedings of the 23rd annual ACM SIGIR, pp. 41-48.
Mason, et al. “Boosting Algorithms as Gradient Descent” (2000) NIPS 7 pages.
Caruana, et al. “Using the Future to ‘Sort Out’ the Present: Rankprop and Multitask Learning for Medical Risk Evaluation” (1996) NIPS, pp. 959-965.
Crammer, et al. “Pranking with Ranking” (2001) NIPS, 7 pages.
Baum, et al. “Supervised Learning of Probability Distributions by Neural Networks” (1988) Neural Information Processing Systems, pp. 52-61.
Orr, et al. “Neural Networks: Tricks of the Trade”, Springer, 1998.
Refregier, et al. “Probabilistic Approach for Multiclass Classification with Neural Networks” (1991) Proceedings of the 1991 International Conference on Artificial Neural Networks (ICANN-91) 5 pages.
Herbrich, et al. “Large Margin Rank Boundaries for Ordinal Regression” (2000) Advances in Large Margin Classifiers, pp. 115-132.
Mitchell. “Machine Learning” New York: McGraw-Hill, 1997.

Affiliated with

Also associated with

Rating

Training a learning system with arbitrary cost functions does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Training a learning system with arbitrary cost functions, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Training a learning system with arbitrary cost functions will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-4051951