Reliability measure for a classifier

Details Reliability measure for a classifier Reliability measure for a classifier

2006-02-17

2009-08-18

Donaghue, Larry D (Department: 2441)

Electrical computers and digital processing systems: multicomput

Computer conferencing

Demand based messaging

C706S020000, C706S025000

Reexamination Certificate

active

07577709

ABSTRACT:
In one aspect, a data item is input into a scoring classifier such that the scoring classifier indicates that the data item belongs to a first class. A determination is made as to the amount of retraining of the scoring classifier, based on the data item, that is required to cause the scoring classifier to indicate that the data item belongs to a second class. A reliability measure is determined based on the required amount of retraining and a class of the data item is determined based, at least in part, on the reliability measure.

REFERENCES:
patent: 5493221 (1996-02-01), Wertz
patent: 5671333 (1997-09-01), Catlett et al.
patent: 5796863 (1998-08-01), Lyon
patent: 5805730 (1998-09-01), Yaeger et al.
patent: 5805731 (1998-09-01), Yaeger et al.
patent: 6161130 (2000-12-01), Horvitz et al.
patent: 6199103 (2001-03-01), Sakaguchi
patent: 6330590 (2001-12-01), Cotten
patent: 6421709 (2002-07-01), McCormick
patent: 6507866 (2003-01-01), Barchi
patent: 6732149 (2004-05-01), Kephart
patent: 6823323 (2004-11-01), Forman et al.
patent: 6829635 (2004-12-01), Townshend
patent: 6976207 (2005-12-01), Ruján et al.
patent: 2002/0116463 (2002-08-01), Hart
patent: 2002/0116641 (2002-08-01), Mastrianni
patent: 2002/0147754 (2002-10-01), Dempsey
patent: 2002/0181703 (2002-12-01), Logan
patent: 2002/0199095 (2002-12-01), Bandini
patent: 2004/0083270 (2004-04-01), Heckerman et al.
patent: 2005/0021290 (2005-01-01), Velipasaoglu et al.
Apte et al., Automated Learning of Decision Rules for Text Categorization, ACM Transactions on Information Systems, vol. 12, No. 3, Jul. 1994, pp. 233-251.
Beitzel et al., Evaluation of Filtering Current News Search Results, ACM SIGIR conference on Research and development in information retrieval, 2004, pp. 1-2.
Bennett et al., Probabilistic Combination of Text Classifiers Using Reliability Indicators: Models and Results, Proceedings of 25thAnnual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2002, pp. 207-214.
Bennett, Using Asymmetric Distributions to Improve Text Classifier Probability Estimates, Proceedings of the 26thannual international ACM SIGIR Conference on Research and Development in Information Retrieval, 2003, pp. 111-118.
Bradley, The Use of the Area Under the ROC Curve in the Evaluation of Machine Learning Algorithms, Pattern Recognition, Vo. 30, No. 7, 1997, pp. 1145-1159.
Chowdhury et al., Collection Statistics for Fast Duplicate Document Detection, ACM Transactions on Information Systems, vol. 20, No. 2, 2002, pp. 171-191.
Cleverdon, The Significance of the Cranfield Tests on Indexing Language, Proceedings of the 14thInternational Conference on Research and Development in Information Retrieval, 1991, pp. 3-12.
Cohen et al., Context-Sensitive Learning Methods for Text Categorization, Proceedings of SIGIR-96, 19thACM International Conference on Research and Development in Information Retrieval, 1996, pp. 307-315.
Cover et al., Elements of Information Theory, John Wiley & Sons, Inc., 1991, pp. 1-8.
Cronen-Townsend et al., Predicting Query Performance, Proceedings of SIGR, 2002, pp. 299-306.
Domingos et al., On the Optimality of the Simple Bayesian Classifier Under Zero-One Loss, Machine Learning, vol. 29, 1997, pp. 103-130.
Domingos, Context-Sensitive Feature Selection for Lazy Learners, Artificial Intelligence Review, vol. 11, 1997, pp. 1-24.
Elkan, The Foundations of Cost-Sensitive Learning, Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, 2001, pp. 973-978.
Fleuret, Fast Binary Feature Selection With Conditional Mutual Information, Journal of Machine Learning Research 5, 2004, pp. 1531-1555.
Forman, An Extensive Empirical Study of Feature Selection Metrics for Text Classification, Journal of Machine Learning Research 3, 2003, pp. 1290-1305.
Friedman et al., Bayesian Network Classifiers, Machine Learning, vol. 29, 1997, pp. 131-163.
Graham, A Plan for Spam, http://www.paulgraham.com/spam.html, 2002, pp. 1-15.
Han et al., Centroid-Based Document Classification: Analysis and Experimental Results, Principles of Data Mining and Knowledge Discovery, 2000, pp. 424-431.
Harmon, Overview of the First Text Retrival Conference (TREC-1), Computer Systems and Technology, National Institute of Technology and Standards, 1992, pp. 1-20.
Khoavi et al., Wrappers for Feature Subset Selection, Artificial Intelligence, 1997, pp. 1-43.
Kolcz, Imptoved Naïve Bayes for Extreme Misclassification Costs, High Precision/Recall, abstract submitted Feb. 18, 2005.
Kolcz et al., Improved Naïve Bayes for Extreme Misclassification Costs, High Precision/Recall, abstract submitted May 2005.
Kolcz, Local Scarcity Control for Naïve Bayes With Extreme Misclassification Costs, submitted Feb. 28, 2005.
Koller et al., Toward Optional Feature Selection, International Conference on Machine Learning, 1996, pp. 284-292.
Kukar, Transductive Reliability Estimation for Medical Diagnosis, Artificial Intelligence in Medicine, vol. 29; 2003, pp. 81-106.
Lewis et al., A New Benchmark Collection for Text Categorization Research, Journal of Machine Learning Research, vol. 5, 2004, pp. 361-397.
Webb et al., Adjusted Probability Naive Bayesian Induction, 11thAustralian Joint Conference on Artificial Intelligence, 1998, pp. 1-11.
Wu et al., A Refinement Aproach to Handling Model Misfit in Text Categorization, ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2002, pp. 1-10.
Wu et al., The Anatomy of a Multimodal Information Filter, Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2003, pp. 462-471.
Zadrozny et al., Obtaining Calibrated Probability Estimates From Decision Trees and Naive Bayesian Classifiers, Eighteenth International Conference on Machine Learning, 2001, pp. 1-8.
Zadrozny, et al., Transforming Classifier Scores in to Accurate Multiclass Probability Estimates, Eighth International Conference on Knowledge Discovery and Data Mining, 2002, pp. 1-6.
Zhang et al., Probabilistic Score Estimation With Piecewise Logistic Regression, Proceedings of the International Conference On Machine Learning, 2004, pp. 1-8.
Lewis et al., Training Algorithms for Linear Text Classifiers, Proceedings of SIGIR-96, 19thACM International Conference on Research and Development in Information Retrieval, 1996, pp. 298-306.
Lewis, Naïve (Bayes) at Forty: The Independence Assumption in Information Retrieval, 10thEuropean Conference on Machine Learning, 1998, pp. 4-15.
Manmatha et al., A Critical Examination of TDT's Cost Function, Proceedings of the 25thAnnual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2002, pp. 403-404.
Manning et al., Foundations of Statistical Natural Language Processing, MIT Press, 1999, pp. 1-23.
McCallum et al., A Comparison of Event Models for Naïve Bayes Text Classification, Proceedings of the AAAI-98 Workshop on Learning for Text Categorization, 1998, pp. 1-8.
Mladenic et al., Feature Selection for Unbalanced Class Distribution and Naive Bayes, Proceedings for the 16thInternational Conference on Machine Learning, 1999, pp. 258-267.
Mladenic et al., Feature Selection Using Linear Classifier Weights: Interaction With Classification Models, Proceedings of SIGIR-04, 2004, pp. 234- 241.
Moschitti, A Study on Optimal Parameter Tuning for Rocchio Text Classifier, Proceedings of ECIR-03, 25thEuropean Conference on Information Retrieval, 2003, pp. 420-435.
P.N. Bennett, Assessing the Calibration of Naïve Bayes Posterior Estimates, School of Computer Science, Carnegie Mellon University, Sep. 12, 2000., pp. 1-10.
Pavlov et al., Document Preprocessing for Naïve Bayes Classification and Clustering With Mixture of Multinomials, Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2004, pp. 1-6.
Platt, Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods, Advances in Large Margin Classifiers, MIT Press, 1999, pp. 61-74.
Provost et al., Robust Classification for Imprecise Environments, Machine L

Affiliated with

Also associated with

Rating

Reliability measure for a classifier does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Reliability measure for a classifier, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Reliability measure for a classifier will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-4096386