Gradient learning for probabilistic ARMA time-series models

Details Gradient learning for probabilistic ARMA time-series models Gradient learning for probabilistic ARMA time-series models

2004-12-14

2008-09-02

Phan, Thai (Department: 2128)

Data processing: structural design, modeling, simulation, and em

Modeling by mathematical expression

C702S182000, C702S183000, C702S184000, C706S021000

Reexamination Certificate

active

07421380

ABSTRACT:
The subject invention leverages the conditional Gaussian (CG) nature of a continuous variable stochastic ARMAxptime series model to efficiently determine its parametric gradients. The determined gradients permit an easy means to construct a parametric structure for the time series model. This provides a gradient-based alternative to the expectation maximization (EM) process for learning parameters of the stochastic ARMAxptime series model. Thus, gradients for parameters can be computed and utilized with a gradient-based learning method for estimating the parameters. This allows values of continuous observations in a time series to be predicted utilizing the stochastic ARMAxptime series model, providing efficient and accurate predictions.

REFERENCES:
patent: 5544281 (1996-08-01), Maruoka et al.
patent: 5809499 (1998-09-01), Wong et al.
patent: 5835682 (1998-11-01), Broomhead et al.
patent: 5949678 (1999-09-01), Wold et al.
patent: 6125105 (2000-09-01), Edwards et al.
patent: 6336108 (2002-01-01), Thiesson et al.
patent: 6345265 (2002-02-01), Thiesson et al.
patent: 6363333 (2002-03-01), Deco et al.
patent: 6408290 (2002-06-01), Thiesson et al.
patent: 6496816 (2002-12-01), Thiesson et al.
patent: 6529891 (2003-03-01), Heckerman
patent: 6532454 (2003-03-01), Werbos
patent: 6560586 (2003-05-01), Liang et al.
patent: 6574587 (2003-06-01), Waclawski
patent: 6735580 (2004-05-01), Li et al.
patent: 6742003 (2004-05-01), Heckerman et al.
patent: 6778929 (2004-08-01), Egi
patent: 6807537 (2004-10-01), Thiesson et al.
patent: 6853920 (2005-02-01), Hsiung et al.
patent: 6882992 (2005-04-01), Werbos
patent: 6928398 (2005-08-01), Fang et al.
patent: 6987865 (2006-01-01), Szeliski et al.
patent: 7092457 (2006-08-01), Chugg et al.
patent: 7139703 (2006-11-01), Acero et al.
patent: 2002/0072882 (2002-06-01), Kruger et al.
patent: 2003/0039867 (2003-02-01), Meek et al.
patent: 2003/0046038 (2003-03-01), Deligne et al.
patent: 2003/0055614 (2003-03-01), Pelikan et al.
patent: 2003/0065409 (2003-04-01), Raeth et al.
patent: 2003/0176931 (2003-09-01), Pednault et al.
patent: 2004/0068199 (2004-04-01), Echauz et al.
patent: 2004/0068332 (2004-04-01), Ben-Gal et al.
patent: 2004/0101048 (2004-05-01), Paris
patent: 2004/0260664 (2004-12-01), Thiesson et al.
patent: 2005/0015217 (2005-01-01), Weidl et al.
patent: 2005/0096873 (2005-05-01), Klein
patent: 2006/0074558 (2006-04-01), Wiliiamson et al.
patent: 2006/0129395 (2006-06-01), Thiesson et al.
patent: 2006/0247900 (2006-11-01), Brocklebank
patent: 2007/0150077 (2007-06-01), Bocharov et al.
patent: 2008/0010043 (2008-01-01), Thiesson et al.
Bo Thiesson, et al., ARMA Time-Series Modeling With Graphical Models, Proceedings of the Twentieth Conference on Uncertainty in Artificial Intelligence, (2004), pp. 552-560. AUAI Press.
Bach, F.G., et al., Learning Graphical Models For Stationary Time Series, (2004), IEEE Transactions On Signal Processing, to appear.
Cooper, G., et al., A Model For Temporal Probabilistic Reasoning (Technical Report KSL-88-30), (1998), Stanford University, Section On Medical Informatics, Stanford, California.
Ghahramani, Z., Learning Dynamic Bayesian Networks. In Adaptive Processing Of Sequences And Data Structures. Lecture Notes in Artificial Intelligence, (1998), p. 168-197, Springer-Verlag, Berlin.
Lauritzen, S. L., et al., Stable Local Computation With Conditional Gaussian Distributions, Statistics and Computing, (2001), pp. 11, 191-203.
Meek, C., Chickering, et al., Autoregressive Tree Models For Time-Series Analysis, Proceedings Of the Second Internationsl SIAM Conference On Data Mining, (2002), pp. 229-244, Arlington, VA: SIAM.
Bo Thiesson, et al., Efficient gradient computation for conditional Gaussian models. In Ghahramani, Z. and Cowell, R., editors, Proceedings of 10th Int'l Workshop on Artificial Intelligence and Statistics. The Society of Artificial Intelligence and Statistics, Jan. 2005.
Box, G.E.P., et al, Time Series Analysis : Forecasting & Control (3rd Edition), 1994, Prentice Hall, New Jersey.
Golub, G. H, et al., Matrix computations, 1996, The Johns Hopkins University Press, London.
Reinsel, G. C., Elements Of Multivariate Time series Analysis, 2003, Springer-Verlag, New York.
Dempster, A. P., et al., Maximum Likelihood From Incomplete Data Via The EM Algorithm, 1997, Journal of the Royal Statistical Society, B39, pp. 1-38.
Jones R. H., Maximum Likelihood Fitting Of ARMA Models To Time series With Missing Observation, 1980, Technometrics, vol. 22, pp. 389-395.
Penzer, J., et al., The Exact Likelihood Of An autoregressive-Moving Average Model With Imcomplete Data, (1997), Biometrika, vol. 84, pp. 919-928.
Kai Ming Ting, et al., Theory Combination: an alternative to Data Combination, 1996, 22 pages.
Sam-Joo Doh, et al., Inter-Class MLLR For Speaker Adaptation, Department of Electrical and Computer Engineering and School of Computer Science, Carnegie Mellon University, 2000, 4 pages.
Oliver Siohan, et al., Structural Maximum a Posteriori Linear Regression For Fast HMM Adaptation, Multimedia Communications Research Lab Bell Laboratories—Lucent Technologies, 2000, 8 pages.
Andrew W. Moore, et al. Efficient Locally Weighted Polynomial Regression Predictions, 1997, 9 pages.
Aram Karalic. Employing Linear Regression in Regression Tree Leaves, European Conference on Artificial Intelligence. 1992, 2 pages.
Hugh Chipman, et al. Bayesian Treed Models, Feb. 2001, 29 pages.
George Kapetanios, Threshold Models for Trended Time Series, 1999, 32 pages.
Felix A. Gers, et al. Applying LSTM to Time Series Predictable Through Time-Window Approaches, 2001, 21 pages.
David Maxwell Chickering, et al. A Bayesian Approach to Learning Bayesian Networks with Local Structure, Aug. 1997, 19 pages.
Neil A. Gershenfeld, et al. The Future of Time Series, 1993, pp. i-iii, 1-70, 569-582.
Craig F. Ansley. An algorithm for the exact likelihood of a mixed autoregressive-moving average process, 1979, vol. 66, pp. 59-65, Biometrika.
John Binder, et al. Adaptive probabilistic networks with hidden variables; Machine Learning, 1997, 213-244, 29.
Bo Thiesson. Score and Information for Recursive Exponential Models with Incomplete Data. In Geiger, D. and Shenoy, P.P., editors. Proceedings of the 13th Conference on Uncertainty in Al, 1997 pp. 453-463. Morgan Kaufmann Publishers.
Berzuini, et al. “A Unified Approach for Modelling Longitudinal and Failure Time Data, with Application in Medical Monitoring.” IEEE Trans. On Pattern Analysis and Machine Intelligence. Feb. 1996, vol. 18, issue 2, pp. 109-123.
Chickering, et al. “Efficient Determination Of Dynamic Split Points in a Decision Tree.” Proc. IEEE Int'l. Conf. on Data Mining. Nov. 29-Dec. 2, 2001. pp. 91-98.
Chipman, et al. Bayesian Treed Models, Feb. 2001. pp. 1-29.
Coggers. Modern Regression Methods: A Comparative Discussion, Apr. 19, 2001, pp. 1-11.
Gers. Applying LSTM to Time Series Predictable Through Time Window Approaches, 2001, pp. 1-8.
Heckerman, et al. “Casual Independence for Probability Assessment and Interference Using Bayesian Networks.” IEEE Transactions on Systems, Man and Cybernetics. Nov. 1996, vol. 26, Issue 6, pp. 826-831.
Heckerman, et al. “Dependency Networks on Inference, Collaboration Filtering and Data Visualization.” The Journal of Machine Learning Research. Sep. 2001, vol. 1, pp. 49-75.
Howell Tong, Threshold Models in Non-Linear Time Series Analysis, 1983, 323 pages, Springer-Verlag, New York, NY.
Kapetanios. A Threshold Model for Trended Time Series, 1999, pp. 1-32.
Meek, et al. Autoregressive Tree Models for Time Series Analysis. Jan. 10, 2002. http://research.microsoft.com/˜meek/papers/dmart.ps&pub=63.
Meek. “Annotated Bibliography of Papers by Christopher Meek.” Mar. 2005, printed Dec. 28, 2006. http://research.microsoft.com/˜meek/mypapers.htm.
P. A. W. Lewis, et al., Modeling Time Series by Using Multivariate Adaptive Regression Splines (MARS), Time Series Prediction, 1994, pp. 297-318, Addison Wesley, New York, NY.
U.S. Appl. No. 11/005,148, filed Dec. 6, 2004, Thiesson,

Affiliated with

Also associated with

Rating

Gradient learning for probabilistic ARMA time-series models does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Gradient learning for probabilistic ARMA time-series models, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Gradient learning for probabilistic ARMA time-series models will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3991468