Electrical transmission or interconnection systems – Nonlinear reactor systems – Parametrons
Patent
1989-07-26
1990-05-15
Hudspeth, David
Electrical transmission or interconnection systems
Nonlinear reactor systems
Parametrons
364513, 364807, G06G 7184
Patent
active
049260642
ABSTRACT:
A method and apparatus are disclosed for implementing a neural network having a sleep mode during which capacitively stored synaptic connectivity weights are refreshed. Each neuron outputs an analog activity level, represented in a preferred embodiment by the frequency of digital pulses. Feed-forward synaptic connection circuits couple the activity level outputs of first level neurons to inputs of second level neurons, and feed-back synaptic connection circuits couple outputs of second level neurons to inputs of first level neurons, the coupling being weighted according to connectivity weights stored on respective storage capacitors in each synaptic connection circuit. The network learns according to a learning algorithm under which the connections in both directions between a particular first level neuron and a particular second level neuron are strengthened to the extent of concurrence of high activity levels in both the first and second level neurons, and weakened to the extent of concurrence of a high activity level in the second level neuron and a low activity level in the first level neuron. The network is put to sleep by disconnecting all environmental inputs and providing a non-specific low activity level signal to each of the first level neurons. This causes the network to randomly traverse its state space with low intensity resonant firings, each state being visited with a probability responsive to the initial connectivity weights of the connections which abut the second level neuron representing such state. Refresh is accomplished since the learning algorithm remains active during sleep. Thus, the sleep refresh mechanism enhances the contrast in the connectivity terrain and strengthens connections that would otherwise wash out due to lack of visitation while the system is awake. A deep sleep mechanism is also provided for preventing runaway strengthening of favored states, and also to encourage Weber Law compliance.
REFERENCES:
patent: 3273125 (1966-09-01), Jakowitz
patent: 3496382 (1970-02-01), Hendrix
patent: 3950733 (1976-04-01), Cooper et al.
patent: 4660166 (1987-04-01), Hopfield
patent: 4731747 (1988-03-01), Denker
patent: 4807168 (1989-02-01), Moopenn et al.
Hoffmann and Benson, "Neurons With Hysteresis Form a Network That Can Learn Without Any Changes in Synaptic Connection Strengths", Conf.-Neural Networks for Computing, Snowbird, UT, U.S.A. 13-16 Apr. 1986.
Kosugi et al, "Considerations on Pattern Separation in Randomly Connected Layered Neuronal Networks", Systems Computers Controls, vol. 6, No. 5, 19785, pp. 68-76 (307/201).
"The Art of Adaptive Pattern Recognition by a Self-Organized Neural Network,": Gail A. Carpenter and Stephen Grossberg, IEEE, Mar. 1988, pp. 77-88.
"Neocognitron: A Hierarchical Neural Network Capable of Visual Pattern Recognition," Kunihiko Fukushima,Neural Networks, vol. 1, 1988, pp. 119-130.
"Implementations of Neural Netwsork Models In Silicon," Stuart Mackie, hans P. Graf, and Daniel B. Schwartz, NATO Advanced Research Workshop, 1987, pp. 467-476.
"A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine," Gail A. Carpenter, Computer Vision, Graphics, and Image Processing, vol. 37, 1987, pp. 54-115.
"The Function of Dream Sleep," Francis Crick and Graeme Mitchison, Nature, vol. 304, 14 Jul. 1983, pp. 111-114.
"`Unlearning` Has A Stabilizing Effect in Collective Memories," J. J. Hopfield, D. I. Feinstein and R. G. Palmer, Nature, vol. 304, Jul. 14, 1983, pp. 158-159.
"Analog VLSI and Neural Systems," Carver A. Mead, pp. 79-96 and 113-117, (preliminary 2 version dated 11/5/87).
Hoffman & Benson, "Neurons With Hysteresis Form a Network That Can Learn Without Any Changes in Synaptic Connection Strengths," Conf. Neural Networks for Computing, Snowbird, UT, U.S.A. 13-16 Apr. 1986.
Kosugi et al. "Considerations on Pattern Separation in Randomly Connected Layered Neuronal Networks," Systems Computers Controls, vol. 6, No. 5, 1975, pp. 68-76 (307/201).
Hudspeth David
Syntonic Systems Inc.
LandOfFree
Sleep refreshed memory for neural network does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Sleep refreshed memory for neural network, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Sleep refreshed memory for neural network will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-622989