Data processing: artificial intelligence – Neural network – Learning task
Reexamination Certificate
2003-08-28
2008-09-02
Vincent, David R. (Department: 2129)
Data processing: artificial intelligence
Neural network
Learning task
C706S046000, C382S155000, C382S181000, C382S224000
Reexamination Certificate
active
07421417
ABSTRACT:
A feature selection technique for support vector machine (SVM) classification makes use of fast Newton method that suppresses input space features for a linear programming formulation of a linear SVM classifier, or suppresses kernel functions for a linear programming formulation of a nonlinear SVM classifier. The techniques may be implemented with a linear equation solver, without the need for specialized linear programming packages. The feature selection technique may be applicable to linear or nonlinear SVM classifiers. The technique may involve defining a linear programming formulation of a SVM classifier, solving an exterior penalty function of a dual of the linear programming formulation to produce a solution to the SVM classifier using a Newton method, and selecting an input set for the SVM classifier based on the solution.
REFERENCES:
patent: 6112195 (2000-08-01), Burges
patent: 6134344 (2000-10-01), Burges
patent: 6327581 (2001-12-01), Platt
patent: 6571225 (2003-05-01), Oles et al.
patent: 6728690 (2004-04-01), Meek et al.
patent: 2002/0165854 (2002-11-01), Blayvas et al.
patent: 2003/0093393 (2003-05-01), Mangasarian et al.
patent: 2003/0115030 (2003-06-01), Ewing
patent: 2003/0167135 (2003-09-01), Ewing
patent: 2005/0105794 (2005-05-01), Fung
patent: 2005/0119837 (2005-06-01), Prakash et al.
patent: 2005/0171923 (2005-08-01), Kiiveri et al.
“Empirical error based optimizationof SVM kernels: application to digit image recognition”, Ayat, N. E., Cheriet, M., Suen, C. Y., Frontiers in Handwriting Recognition, 2002. Proceedings 8thIntl, Aug. 6-8, 2002, pp. 292-297.
“Multiple centrality corrections in a primal-dual method for linear programming”, Jacek Gondzio, Computational Optimization and Applications, vol. 6, No. 2, Sep. 1996, pp. 137-156.
W. H. Wolberg and O.L. Mangasarian, “Multisurface Method of Pattern Separation for Medical Diagnosis Applied to Breat Cytology,” Proc. Natl. Acad. Sci. USA, vol. 87, pp. 9193-9196, Dec. 1990.
Y. -J. Lee and O.L. Mangasarian, “SSVM: A Smooth Support Vector Machine for Classification,” Data Mining Institute Technical Report 99-03, Sep. 1999. Computational Optimization and Applications 20, pp. 1-22, 2001.
Y. -J. Lee, O.L. Mangasarian and W.H. Wolberg, “Breast Cancer Survival and Chemotherapy: A Support Vector Machine Analysis,” Data Mining Institute Technical Report 99-10, Dec. 1999. DIMACS Series in Discrete Mathematics and Computer Science, vol. 55, American Mathematical Society, pp. 1-10, 1999.
N. Cristianini and John Shawe-Taylor, “An Introduction to Support Vector Machines and other Kernel-Based Learning Methods,” Cambridge University Press, 2000, ISBN: 0 521 78019 5, 10 pgs.
O.L. Mangasarian and D.R. Musicant, “Data Discrimination via Nonlinear Generalized Support Vector Machines,” Mathematical Programming Technical Report 99-03, Computer Sciences Department, Unversity of Wisconsin, Madison, Wisconsin, Mar. 1999, 14 pgs.
O.L. Mangasarian and D.R. Musicant,“Successive Overrelaxation for Support Vector Machines,” IEEE Transactions on Neural Networks, vol. 10, No. 5, Sep. 1999, pp. 1032-1037.
A.I. Su et al., “Molecular Classification of Human Carcinomas by Use of Gene Expression Signatures,” Cancer Research 61, Oct. 15, 2001, pp. 7388-7393.
Barlogie, B., Cussens, J., Hardin, J., Page, D., Shaughnessy, J., Waddell, M., and Zhan, F., “Comparative data mining for microarrays: A case study based on multiple myeloma,” Technical Report 1453, Computer Sciences Department, University of Wisconsin, Madison, Nov. 2002.
Bradley, P. S., and Mangasarian, O. L., “Feature Selection via Concave Minimization and Support Vector Machines,” Machine Learning Proceedings of the Fifteenth International Conference (ICML '98) pp. 82-90, San Francisco, CA 1998.
Fung, G. and Mangasarian, O. L., “A Feature Selection Newton Method for Support Vector Machine Classification,” Data Mining Institute Technical Report 02-03, pp. 1-18, Sep. 2002. Computational Optimization and Applications, to appear.
Fung, G. and Mangasarian, O. L., “Finite Newton Method for Lagrangian Support Vector Machine Classification,” Data Mining Institute Technical Report 02-03, pp. 1-22, Feb. 2002. Computational Optimization and Applications, Neurocomputing 55 on Sep. 2003.
Golub, G. H. and Van Loan, C. F., “Matrix computations,” The John Hopkins University Press, Baltimore, Maryland, 3rdedition, pp. 48-86, 1996.
Lee Y. J. and Mangasarian, O. L., “RSVM: Reduced Support Vector Machines,” Data Mining Institute Technical Report 00-07, Computer Sciences Department, University of Wisconsin, Madison, Wisconsin, pp. 1-17, Jul. 2000.
Mangasarian, O. L., “Generalized Support Vector Machines,” in Advances in Large Margin Classifiers, MIT Press, Cambridge, Massachusetts, Bartlett, P., Scholkopf, B., Schuurmans, D., and Smola, A., editors, pp. 135-146, 2000.
Mangasarian, O. L., “Parallel Gradient Distribution in Unconstrained Optimization,” SIAM Journal on Control and Optimization, 33(6):1916-1925, 1995.
Mangasarian, O. L. and Musicant, D. R., “Lagrangian Support Vector Machine,” Journal of Machine Learning Research, 1:161-177, 2001.
Odewahn, E. Stockwell, R. Pennington, R. Humphreys, and W. Zumach, “Automated Star/Galaxy Discrimination with Neural Networks,” Astronomical Journal, 103(1): 318-331, 1992.
Burges, Christopher J.C., “A Tutorial on Support Vector Machines for Pattern Recognition,” Data Mining and Knowledge Discovery, vol. 2, No. 2, pp. 121-167, 1998.
Fung, Glen, Mangasarian, Olvi L., “Data Selection for Support Vactor Machines Classifiers,” Data Mining Institute Report 00-02, 7 pgs., Feb. 2000.
Trotter, Matthew, “Support Vector Machines for QSAR Analysis,” Department of Computer Science, University of College London, 25 pgs., undated.
F. Facchinei, Minimization of SC1functions and the Maratos effect. Operations Research Letters, 17:131-137, 1995.
J.B. Hiriart-Urruty, J.J. Strodiot, and V.H. Nguyen, Generalized hessian matrix and second-order optimality conditions for probelms with C1,1data, Applied Mathematics and Optimization, 11:43-56, 1984.
CPLEX, http://www.llog.com/products/cplex, 1 page (last printed Feb. 10, 2004).
S. Lucidi, A new result in the theory and computation of the least-norm solution of a linear program, Journal of Optimization Therory, and Applictions, 55:103-117, 1987.
O.L. Mangasarian, Normal solutions of linear programs, Mathematical Programming Study, 22:206-216, 1984.
O.L. Mangasarian, Arbitrary-norm separting plane, Operations Research Letters, 24:15-23, 1999.
O.L. Mangasarian, A finite Newton method for classification problems, Technical Report 01-11, Data Mining Institute, Computer Sciences Department, University of Wisconsin, pp. 1-17, Dec. 2001.
O.L. Mangasarian and R.R. Meyer, Nonlinear perturbation of linear programs, SIAM Journal on Control and Optimization, 17(6):745-752, Nov. 1979.
O.L. Mangasarian, A Newton Method for Linear Programming, Powerpoint Presentation, Mathematics Department, University of California at San Diego, 21 pgs., Jul. 26, 2002.
O.L. Mangasarian, A Newton Method for Linear Programming, Technical Report 02-02, Data Mining Institute, Computer Sciences Department, University of Wisconsin, pp. 1-20, Mar. 2002. Revised Dec. 2002, Jrnl. of Opti. Theory & Appl. 121,2004, pp. 1-18.
Office Action dated Jul. 25, 2007 for U.S. Appl. No. 10/114,419, filed Apr. 1, 2002, (11 pages).
Fung Glenn M.
Mangasarian Olvi L.
Shumaker & Sieffert P.A.
Tran Mai T.
Vincent David R.
Wisconsin Alumni Research Foundation
LandOfFree
Input feature and kernel selection for support vector... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Input feature and kernel selection for support vector..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Input feature and kernel selection for support vector... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3976938