Data processing: artificial intelligence – Neural network – Structure
Reexamination Certificate
2008-01-22
2008-01-22
Holmes, Michael B. (Department: 2121)
Data processing: artificial intelligence
Neural network
Structure
C706S015000
Reexamination Certificate
active
07321882
ABSTRACT:
A method for the supervised teaching of a recurrent neutral network (RNN) is disclosed. A typical embodiment of the method utilizes a large (50 units or more), randomly initialized RNN with a globally stable dynamics. During the training period, the output units of this RNN are teacher-forced to follow the desired output signal. During this period, activations from all hidden units are recorded. At the end of the teaching period, these recorded data are used as input for a method which computes new weights of those connections that feed into the output units. The method is distinguished from existing training methods for RNNs through the following characteristics: (1) Only the weights of connections to output units are changed by learning—existing methods for teaching recurrent networks adjust all network weights. (2) The internal dynamics of large networks are used as a “reservoir” of dynamical components which are not changed, but only newly combined by the learning procedure—existing methods use small networks, whose internal dynamics are themselves completely re-shaped through learning.
REFERENCES:
patent: 4884229 (1989-11-01), Dekker
patent: 5204872 (1993-04-01), Staib et al.
patent: 5406581 (1995-04-01), Staib et al.
patent: 5408424 (1995-04-01), Lo
patent: 5425130 (1995-06-01), Morgan
patent: 5428710 (1995-06-01), Toomarian et al.
patent: 5465321 (1995-11-01), Smyth
patent: 5479571 (1995-12-01), Parlos et al.
patent: 5486996 (1996-01-01), Samad et al.
patent: 5513098 (1996-04-01), Spall et al.
patent: 5606646 (1997-02-01), Khan et al.
patent: 5613042 (1997-03-01), Chung et al.
patent: 5649065 (1997-07-01), Lo et al.
patent: 5659583 (1997-08-01), Lane
patent: 5671336 (1997-09-01), Yoshida et al.
patent: 5699487 (1997-12-01), Richardson
patent: 5706400 (1998-01-01), Omlin et al.
patent: 5745653 (1998-04-01), Jesion et al.
patent: 5748847 (1998-05-01), Lo
patent: 5764858 (1998-06-01), Sheu et al.
patent: 5781700 (1998-07-01), Puskorius et al.
patent: 5822741 (1998-10-01), Fischthal
patent: 5847952 (1998-12-01), Samad
patent: 5867397 (1999-02-01), Koza et al.
patent: 5909676 (1999-06-01), Kano
patent: 5943659 (1999-08-01), Giles et al.
patent: 5960391 (1999-09-01), Tateishi et al.
patent: 5963929 (1999-10-01), Lo
patent: 5983180 (1999-11-01), Robinson
patent: 5987444 (1999-11-01), Lo
patent: 6092018 (2000-07-01), Puskorius et al.
patent: 6151592 (2000-11-01), Inazumi
patent: 6151594 (2000-11-01), Wang
patent: 6169981 (2001-01-01), Werbos
patent: 6175554 (2001-01-01), Jang et al.
patent: 6212508 (2001-04-01), Sterzing et al.
patent: 6272193 (2001-08-01), Eglit
patent: 6601051 (2003-07-01), Lo et al.
patent: 6751601 (2004-06-01), Zegers
patent: 6839303 (2005-01-01), Handa et al.
patent: 6856577 (2005-02-01), Handa et al.
patent: 0 574 951 (1993-12-01), None
patent: 0 374 604 (1989-12-01), None
Qualitative limitations incurred in implementations of recurrent neural networks Michel, A.N.; Kaining Wang; Derong Liu; Hui Ye; Control Systems Magazine, IEEE vol. 15, Issue 3, Jun. 1995 pp. 52-65.
A Fully-Automated Measurement System for 77-GHz Mixers Wagner, C.; Treml, M.; Hartmann, M.; Stelzer, A.; Jaeger, H.; Instrumentation and Measurement Technology Conference Proceedings, 2007 IEEE May 1-3, 2007 pp. 1-4 Digital Object Identifier 10.1109/IMTC.2007.379425.
Reservoir riddles: suggestions for echo state network research Jaeger, H.; Neural Networks, 2005. IJCNN '05. Proceedings. 2005 IEEE International Joint Conference on vol. 3, Jul. 31-Aug. 4, 2005 pp. 1460-1462 vol. 3 Digital Object Identifier 10.1109/IJCNN.2005.1556090.
Plasmon light scattering of a single gold nanoparticle attached to a single mode optical fiber tip Eah, S.; Jaeger, H.M.; Scherer, N.F.; Xiao-An Lin; Wiederrecht, G.P.; Quantum Electronics Conference, 2004. (IQEC). International 2004 pp. 840-842.
Class E with parallel circuit—a new challenge for high-efficiency RF and microwave power amplifiers Grebennikov, A.V.; Jaeger, H.; Microwave Symposium Digest, 2002 IEEE MTT-S International vol. 3, Jun. 2-7, 2002 pp. 1627-1630 Digital Object Identifier 10.1109/MWSYM.2002.1012169.
A framework for plan execution in behavior-based robots Hertzberg, J.; Jaeger, H.; Zimmer, U.; Morignot, P.; Intelligent Control (ISIC), 1998. Held jointly with IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA) Intelligent Systems and Semiotics (ISAS), Sep. 14-17, 1998 pp. 8-13.
Self-organization of feature detectors in time sequences (SOFT)-a neural network approach to multidimensional signal analysis Wismuller, A.; Jaeger, H.; Ritter, H.; Dersch, D.R.; Palm, G.;Neural Networks Proceedings, 1998, IEEE World Congress on Computational Intelligence. IEEE International Joint Conference on vol. 1, May 1998 pp. 575-580 vol. 1.
Guidance integrated fuzing analysis and simulation Chopper, K.; Jaeger, H.; Stephens, L.; Burdick, D.; Lin, C.-F.; Yang, C.; Control Applications, 1992., First IEEE Conference on Sep. 13-16, 1992 pp. 750-755 vol. 2 Digital Object Identifier 10.1109/CCA.1992.269753.
Seeker-Optimized Guidance Integrated Fuzing Chopper, K.; Jaeger, H.; Stephens, L.; Burdic, D.; Chun Yang; Ching-Fang Lin; Aerospace Control Systems, 1993. Proceedings. The First IEEE Regional Conference on May 25-27, 1993 pp. 559-562.
Improved upper bound on step-size parameters of discrete-time recurrent neural networks for linear inequality and equation systemXue-Bin Liang; Shiu Kit Tso; Circuits and Systems I: Fundamental Theory and Applications, IEEE Transactions on [see also Circuits and Systems I: IEEE Transactions on] vol. 49, Issue 5, May 2002 pp. 692-698.
Global stability analysis of discrete-time recurrent neural networks Barabanov, N.E.; Prokhorov, D.V.; American Control Conference, 2001. Proceedings of the 2001 vol. 6, Jun. 25-27, 2001 pp. 4550-4555 vol. 6 Digital Object Identifier 10.1109/ACC.2001.945696.
Absolute stability conditions for discrete-time recurrent neural networks Liang Jin; Nikiforuk, P.N.; Gupta, M.M.; Neural Networks, IEEE Transactions on vol. 5, Issue 6, Nov. 1994 pp. 954-964 Digital Object Identifier 10.1109/72.329693.
Noisy recurrent neural networks: the discrete-time case Olurotimi, O.; Das, S.; Neural Networks, IEEE Transactions on vol. 9, Issue 5, Sep. 1998 pp. 937-946 Digital Object Identifier 10.1109/72.712165.
A discrete-time multivariable neuro-adaptive control for nonlinear unknown dynamic systems Chih-Lyang Hwang; Ching-Hung Lin; Systems, Man and Cybernetics, Part B, IEEE Transactions on vol. 30, Issue 6, Dec. 2000 pp. 865-877 Digital Object Identifier 10.1109/3477.891148.
Discrete-time algebraic Riccati inequation neuro-LMI solution Tamariz, A.D.R.; Bottura, C.P.; Systems, Man and Cybernetics, 2005 IEEE International Conference on vol. 2, Oct. 10-12, 2005 pp. 1748-1752 vol. 2 Digital Object Identifier 10.1109/ICSMC.2005.1571401.
Discrete-time systems neuro-Riccati equation solution Tamariz, A.D.R.; Bottura, C.R.; Neural Networks, 2005. IJCNN '05. Proceedings. 2005 IEEE International Joint Conference on vol. 4, Jul. 4-31, 2005 pp. 2261-2265 vol. 4.
Stability analysis of discrete-time recurrent neural networks Barabanov, N.E.; Prokhorov, D.V.; Neural Networks, IEEE Transactions on vol. 13, Issue 2, Mar. 2002 pp. 292-303 Digital Object Identifier 10.1109/72.991416.
Recurrent nets that time and count Gers, F.A.; Schmidhuber, J.; Neural Networks, 2000. IJCNN 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on vol. 3, Jul. 24-27, 2000 pp. 189-194 vol. 3 Digital Object Identifier 10.1109/IJCNN.2000.861302.
Training Recurrent Neurocontrollers for Robustness With Derivative-Free Kalman Filter Prokhorov, D. V.; Neural Networks, IEEE Transactions on vol. 17, Issue 6, Nov. 2006 pp. 1606-1616 Digital Object Identifier 10.1109/TNN.2006.880580.
A Recurrent Control Neural Network for Data Efficient Reinforcement Learning Schaefer, A.M.; Udluft, S.; Zimmermann, H.-G.; Approximate Dynamic Programming and Reinforcement Learning, 2007, ADPRL 2007. IEEE International Symposium on Apr. 1-5, 2007 pp. 151-157 Digital Object Identifier 10.1109/ADPRL.20
Birch & Stewart Kolasch & Birch, LLP
Fraunhofer-Gesellschaft zur Foederung der Angewandten Forschung
Holmes Michael B.
LandOfFree
Method for supervised teaching of a recurrent artificial... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method for supervised teaching of a recurrent artificial..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method for supervised teaching of a recurrent artificial... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2795484