Patent
1992-12-22
1995-12-12
Davis, George B.
395 24, 395 22, G06F 1518
Patent
active
054757959
DESCRIPTION:
BRIEF SUMMARY
BACKGROUND OF THE INVENTION
This invention relates to artificial neuron-like devices (hereinafter referred to simply as "neurons") for use in neural processing.
One of the known ways of realising a neuron in practice is to use a random access memory (RAM). The use of RAMs for this purpose dates back a considerable number of years. Recently, a particular form of RAM has been described (see Proceedings of the First IEE International Conference on Artificial Neural Networks, IEE, 1989, No. 313, pp 242-246) which appears to have the potential for constructing neural networks which mimic more closely than hitherto the behaviour of physiological networks. This form of RAM is referred to as a pRAM (probabilistic random access memory). For a detailed discussion of the pRAM attention is directed to the paper identified above. However, a brief discussion of the pRAM is set out below, by way of introduction to the invention.
The pRAM is a hardware device with intrinsically neuron-like behaviour (FIG. 1). It maps binary inputs [5] (representing the presence or absence of a pulse on each of N input lines) to a binary output [4] (a 1 being equivalent to a firing event, a 0 to inactivity). This mapping from {0,1}.sup.N to {0,1} is in general a stochastic function. If the 2.sup.N address locations [3] in an N-input pRAM A are indexed by an N-bit binary address vector u, using an address decoder [6], the output a .epsilon. {0,1} of A is 1 with probability ##EQU1## where i .epsilon. {0,1}.sup.N is the vector representing input activity (and x is defined to be 1-x for any x). The quantity .alpha..sub.u represents a probability. In the hardware realisation of the device .alpha..sub.u is represented as an M-bit integer in the memory locations [3], having a value in the range 0 to 2.sup.M -1 and these values represent probabilities in the range ##EQU2## The .alpha..sub.u may be assigned values which have a neuro-biological interpretation: it is this feature which allows networks of pRAMs, with suitably chosen memory contents, to closely mimic the behaviour of living neural systems. In a pRAM, all 2.sup.N memory components are independent random variables. Thus, in addition to possessing a maximal degree of non-linearity in its response function--a deterministic (.alpha..epsilon.{0,1}.sup.N) pRAM can realise any of the 2.sup.2.spsp.N possible binary functions of its inputs--pRAMs differ from units more conventionally used in neural network applications in that noise is introduced at the synaptic rather than the threshold level; it is well known that synaptic noise is the dominant source of stochastic behaviour in biological neurons. This noise, .nu., is introduced by the noise generator [1]. .nu. is an M-bit integer which varies over time and is generated by a random number generator. The comparator [2] compares the value stored at the memory location being addressed and .nu.. One way of doing this is to add the value stored at the addressed location to .nu.. If there is a carry bit in the sum, i.e. the sum has M+1 bits, a spike representing a 1 is generated on arrival of the clock pulse [7]. If there is no carry bit no such spike is generated and this represents a 0. It can be seen that the probability of a 1 being generated is equal to the probability represented by the number stored at the addressed location, and it is for this reason that the latter is referred to as a probability. It should be noted that the same result could be achieved in other ways, for example by generating a 1 if the value of the probability was greater than .nu.. It can also be noted that because pRAM networks operate in terms of `spike trains` (streams of binary digits produced by the addressing of successive memory locations) information about the timing of firing events is retained; this potentially allows phenomena such as the observed phase-locking of visual neurons to be reproduced by pRAM nets, with the possibility of using such nets as part of an effective `vision machine`.
For information concerning in particular the mathematics of the pRAM
REFERENCES:
patent: 3327291 (1967-06-01), Lee
patent: 3613084 (1971-10-01), Armstrong
patent: 4031501 (1977-06-01), Caruso
patent: 4040094 (1977-08-01), Everett et al.
patent: 4045659 (1977-08-01), Akagawa et al.
patent: 4518866 (1985-05-01), Clymer
patent: 4807168 (1989-02-01), Moopenn et al.
patent: 4809222 (1989-02-01), Van Den Heuvel
patent: 4972363 (1990-11-01), Nguyen et al.
patent: 4996648 (1991-02-01), Jourjine
patent: 5063521 (1991-11-01), Peterson et al.
patent: 5148385 (1992-09-01), Frazier
patent: 5175798 (1992-12-01), Taylor et al.
Milligan et al., "RAM-unit Learning Networks", IEEE First Intl. Conf. Neural Networks, Jun. 1987, pp. II-557 to II-565.
Parallel Processing in Neural Systems and Computers, "Training Stretegies for probabilistic RAMS" pp. 161-164, Mar. 19-21, 1990.
New Electronics vol. 23, No. 1, Jan. 1990, by Boothroyd: "Ram Chips Build Simple Quick Thinking Networks" see p. 17.
IEEE First International Conference on Neural Networks, vol. 3. Jun. 21, 1987, by Nguyen: "Stochastic Processing in Neural Netrwork Application", pp. Cover Sheet and III-283 to III-291.
Parellel Processing in Neural Systems and Computers; Mar. 19-21, 1990, by Massen: "Why Do Neural Network Researches Ignore Stochastic Computers?", pp. 23-26.
J. G. Taylor, "Spontaneous Behavior in Neural Networks, " J. Theor. Biol. 36, pp. 513-528 (1972).
W. K. Kan and I. Aleksander, "A Probabilistic Logic Neuron Network for Associative Learning", IEEE Proceedings of the First Int'l. Conference on Neural Networks, 1978, pp. 541-548 (8 pages).
D. Gorse and J. G. Taylor, "On the Equivalence and Properties of Noisy Neural and Probaslistic RAM Nets", Physics Letters A, vol. 131, No. 6, pp. 326-332, 1988.
I. Aleksander, "The Logic of Connectionist Systems", Neural Computing Architectures, ed. I. Aleksander, MIT Press, 1989, pp. 133-155 (23 pages).
D. Gorse and J. G. Taylor, "An Analysis of Noisy RAM and Neural Nets", Physica D34, pp. 90-114, 1989.
D. Gorse and J. G. Taylor, "Towards a Hardware Realisable Model of the Neuron", Models of Brain Function, ed. Rodney M. J. Cotterill, CUP (1989), pp. 465-482.
T. G. Clarkson, D. Gorse and J. G. Taylor, "Hardware Realisable Models of Neural Processing", Proceedings of the First IEEE International Conference on Artificial Neural Networks, 1989, pp. 242-246.
C. E. Myers, "Output Functions for Probabilistic Logic Nodes", Proceedings of the First IEEE International Conference on Artificial Neural Networks, 1989, pp. 310-314.
D. Gorse, "A New Model of the Neuron", New Developments in Neural Computing, eds. J. G. Taylor and C. L. T. Mannion, Adam Hilger (1989), pp. 111-118.
D. Martland, "Adaptation of Boolean Networks Using Back-Error Propagation", Brunel University Department of Computer Science Preprint, 1989, (7 pages).
P. C. Bressloff and J. G. Taylor, "Random Iterative Networks", Phys. Rev. A41, pp. 1126-1137 (1990)(22 pages).
D. Gorse and J. G. Taylor, "A General Model of Stochastic Neural Processing", Biol. Cybern, 63, pp. 299-306, 1990.
D. Gorse and J. G. Taylor, "Training Strategies for Probabilistic RAMs", Parallel Processing in Neural Systems and Computers eds. R. Eckmiller, G. Hartmann and G. Hauske, North-Holland, 1990, pp. 161-164 (4 pages).
D. Gorse and J. G. Taylor, "Training Strategies for Probabilistic RAMs", Proceedings of the Neural Computing Meeting (NCM '90), London, Apr. 1990, (7 pages).
D. Gorse and J. G. Taylor, "Hardware Realisable Learning Algorithms", Proceedings of INNC-90-Paris, Kluwer, 1990, pp. 821-824 (4 pages).
C. Myers, "Reinforcement Learning When Results are Delayed and Interleaved in Time", Proceedings of INN-C-90-Paris, Kluwer, 1990, pp. 860-863.
D. Gorse and J. G. Taylor, "Reinforcement Training Strategies for Probabilistic RAMs", Proceedings of Neuronet '90, Prague, Sep. 1990, pp. 98-100 (3 pages).
W. K. Kan, H. K. Wong and N. M. Law, "Non-Overlapped Trees of Probabilistic Logic Neurons", IEEE Region 10 Conference on Computer and Commiunications Systems, Hong Kong, Sep. 1990, pp. 37-39.
Clarkson Trevor G.
Gorse Denise
Taylor John G.
Davis George B.
King's College London
University College London
LandOfFree
Neural processing devices for handling real-valued inputs does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Neural processing devices for handling real-valued inputs, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Neural processing devices for handling real-valued inputs will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-1367438