Bayesian approach for learning regression decision graph...

Details Bayesian approach for learning regression decision graph... Bayesian approach for learning regression decision graph...

2002-03-19

2010-02-09

Rodriguez, Paul L (Department: 2123)

Data processing: structural design, modeling, simulation, and em

Modeling by mathematical expression

C706S050000

Reexamination Certificate

active

07660705

ABSTRACT:
Methods and systems are disclosed for learning a regression decision graph model using a Bayesian model selection approach. In a disclosed aspect, the model structure and/or model parameters can be learned using a greedy search algorithm applied to grow the model so long as the model improves. This approach enables construction of a decision graph having a model structure that includes a plurality of leaves, at least one of which includes a non-trivial linear regression. The resulting model thus can be employed for forecasting, such as for time series data, which can include single or multi-step forecasting.

REFERENCES:
patent: 5544281 (1996-08-01), Maruoka et al.
patent: 5809499 (1998-09-01), Wong et al.
patent: 5835682 (1998-11-01), Broomhead et al.
patent: 5949678 (1999-09-01), Wold et al.
patent: 6125105 (2000-09-01), Edwads et al.
patent: 6336108 (2002-01-01), Thiesson et al.
patent: 6345265 (2002-02-01), Thiesson et al.
patent: 6363333 (2002-03-01), Deco et al.
patent: 6408290 (2002-06-01), Thiesson et al.
patent: 6496816 (2002-12-01), Thiesson et al.
patent: 6529891 (2003-03-01), Heckerman
patent: 6532454 (2003-03-01), Werbos
patent: 6560586 (2003-05-01), Liang et al.
patent: 6574587 (2003-06-01), Waclawski
patent: 6735580 (2004-05-01), Li et al.
patent: 6742003 (2004-05-01), Heckerman et al.
patent: 6778929 (2004-08-01), Egi
patent: 6807537 (2004-10-01), Thiesson et al.
patent: 6853920 (2005-02-01), Hsiung et al.
patent: 6882992 (2005-04-01), Werbos
patent: 6928398 (2005-08-01), Fang et al.
patent: 6987865 (2006-01-01), Szeliski et al.
patent: 7092457 (2006-08-01), Chugg et al.
patent: 7139703 (2006-11-01), Acero et al.
patent: 2002/0072882 (2002-06-01), Kruger et al.
patent: 2003/0039867 (2003-02-01), Meek et al.
patent: 2003/0046038 (2003-03-01), Deligne et al.
patent: 2003/0055614 (2003-03-01), Pelikan et al.
patent: 2003/0065409 (2003-04-01), Raeth et al.
patent: 2003/0176931 (2003-09-01), Pednault et al.
patent: 2004/0068199 (2004-04-01), Echauz et al.
patent: 2004/0068332 (2004-04-01), Ben-Gal et al.
patent: 2004/0101048 (2004-05-01), Paris
patent: 2004/0260664 (2004-12-01), Thiesson et al.
patent: 2005/0015217 (2005-01-01), Weidl et al.
patent: 2005/0096873 (2005-05-01), Klein
patent: 2006/0074558 (2006-04-01), Williamson et al.
patent: 2006/0129395 (2006-06-01), Thiesson et al.
patent: 2006/0247900 (2006-11-01), Brocklebank
patent: 2007/0150077 (2007-06-01), Bocharov et al.
patent: 2008/0010043 (2008-01-01), Thiesson et al.
Heckerman, D. and Breese, J.S. “Causal Independence for Probability Assessment and Inference Using Bayesian Networks.” IEEE Transactions on Systems, Man and Cybernetics. Nov. 1996. vol. 26, Issue 6. pp. 826-831.
Chickering, D.M. et al. “Efficient Determination of Dynamic Split Points in a Decision Tree.” Proc. IEEE Int'l Conf. on Data Mining. Nov. 29-Dec. 2, 2001. pp. 91-98.
Heckerman, D. et al. “Dependency Networks for Inference, Collaborative Filtering, and Data Visualization.” The Journal of Machine Learning Research. Sep. 2001. vol. 1. pp. 49-75.
Berzuini, C. and C. Larizza. “A Unified Approach for Modeling Longitudinal and Failure Time Data, with Application in Medical Monitoring.” IEEE Transactions on Pattern Analysis and MAchine Intelligence. Feb. 1996. vol. 18, Issue 2, pp. 109-123.
Meek, C. “Annotated Bibliography of Papers by Christopher Meek.” Mar. 2005. Printed Dec. 28, 2006. http://research.microsoft.com/˜meek/mypapers.htm.
Meek, C. et al. “Autoregressive Tree Models for Time-Series Analysis.” Jan 10, 2002. http://research.microsoft.com/˜meek/papers/dmart.ps&pub=63.
U.S. Appl. No. 10/463,145, filed Jun. 17, 2003, Thiesson, et al.
U.S. Appl. No. 11/319,894, filed Dec. 28, 2005, Bocharov, et al.
Craig F. Ansley, “An algorithim for the exact likelihood of a mixed autoregressive-moving average process”, 1979, pp. 59-65, Biometrika, printed in Greet Britain.
Neil A. Gershenfeld and Andreas S. Weigend, The Future of Time Series: Learning and Understanding, Time Series Prediction, 1994, pp. 1-70, Addison Wesley, New York, NY.
P. A. W. Lewis, et al., Modeling Time Series by Using Multivariate Adaptive Regression Splines (MARS), Time Series Prediction, 1994, pp. 297-318, Addison Wesley, New York, NY.
Howell Tong, Threshold Models in Non-Linear Time Series Analysis, 1983, 323 pages, Springer-Verlag, New York, NY.
Karalic, Aram. “A Employing Linear Regression in Regression Tree Leaves”,European Conference on Artificial Intelligence, 1992, pp. 1-2.
Chipman, Hugh, et al. ABayesian Treed Models, Feb. 2001, pp. 1-29.
Kapetanios, George. A Threshold Models for Trended Time Series,1999, pp. 1-32.
Gers. Felix A. Applying LSTM to Time Series Predictable Through Time-Window Approaches, 2001, pp. 1-8.
Cogger, Kenneth O. AModern Regression Methods: A Comparative Discussion@, Apr. 19, 2001, pp. 1-11.
Chickering, David Maxwell. AA Bayesian Approach to Learning Bayesian Networks with Local Structure@, Aug. 1997. pp. 1-19.
Bach, F.G., et al., Learning Graphical Models For Stationary Time Series, (2004), IEEE Transactions On Signal Processing, to appear, 11 Pages.
Bo Thiesson, et al. ARMA Time Series Modeling with Graphical Models. Proceedings of the Twentieth Conference on Uncertainity in Arificial Intelligence, 2004, pp. 552-560. AUAI Press.
Bo Thiesson, et al. Efficient Gradient Computaion for Conditional Gaussian Models. Proceedings of 10th Int'l Workshop on Artificial Intelligence and Statistics. The Society for Artificial Intelligence and Statistics, Jan. 2005, 8 Pages.
Bo Thiesson. Score and Information for Recursive Exponential Modelswith incomplete Data. Proceedings of the Thirteenth Conference on uncertainity in AI, 1997, pp. 453-463. Morgan Kaufmann Publishers.
Thomas Dean and Keiji Kanazawa, Probabilistic Temporal Reasoning, Technical Report, May 1988, Brown University, pp. 524-528.
Cooper, G., et al., A Model For Temporal Probabilistic Reasoning (Technical Report KSL-88-30), (1988), Stanford University, Section On Medical Informatics, Stanford, California, 24 Pages.
Dempster, et al. Maximum Likelyhood for Incomplete Data Via the EM Algorithm, 1997, Journal of the Royal Statistical Soceity, B39, pp. 1-38.
Jones. Maximum Likelyhood Fitting of ARMA Models to Time Series with Missing Observations. 1980, Technometrics, pp. 389-395.
Matthias Seeger. Bayesian Gaussian Process Models: PAC-Bayesian Generalization Error Bounds and Sparse Approximation, Jul. 2003, 327 Pages.
Ghahramani, Z., Learning Dynamic Bayesian Networks. In Adaptive Processing Of Sequences And Data Structures. Lecture Notes in Artificial Intelligence, (1998), p. 168-197, Springer-Verlag, Berlin.
Kenneth O. Cogger, Modern Regression Methods: A Comparative Discussion, peakconsulitng.com/coggerdc.pdf, 2001,12 pages, Peak Consulting.
Lauritzen, S. L., et al., Stable Local Computation With Conditional Gaussian Distributions, Statistics and Computing, (2001), pp. 11, 191-203.
John Binder, et al. Adaptive Probabilistic Networks with Hidden Variables, Machine Learning, 1997, pp. 213-244.
“Theory Combination: An Alternative to Data Combination”; Kai Ming Ting; pp. 1-22.
“Inter-Class Mllr for Speaker Adaptation”; Sam-Joo Doh, et al.; Carnegie Mellon University.
“Structural Maximum a Posteriori Linear Regression for Fast HMM Adaption”; Olivier Siohan, et al.; Bell Laboratories—Lucent Technologies.
“Efficient Locally Weighted Polynomial Regression Predictions”; Andrew W. Moore, et al.; Carnegie Mellon University.
“Theory Combination: An Alternative to Data Combination”; Kai Ming Ting; pp. 1-22, Oct. 1996.
“Inter-Class MIlr for Speaker Adaptation”; Sam-Joo Doh, et al.; Carnegie Mellon University, Jun. 2000.
“Structural Maximum a Posteriori Linear Regression For Fast HMM Adaption”; Olivier Siohan, et al.; Bell Laboratories—Lucent Technologies, Sep. 2000.
“Efficient Locally Weighted Polynomial Regression Predictions”; Andrew W. Moore, et al.; Ca

Affiliated with

Also associated with

Rating

Bayesian approach for learning regression decision graph... does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Bayesian approach for learning regression decision graph..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Bayesian approach for learning regression decision graph... will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-4217973