Low-voltage, very-low-power conductance mode neuron

Details Low-voltage, very-low-power conductance mode neuron Low-voltage, very-low-power conductance mode neuron

1999-12-14

2001-07-31

Powell, Mark R. (Department: 2122)

Data processing: artificial intelligence

Neural network

Reexamination Certificate

active

06269352

ABSTRACT:

TECHNICAL FIELD
The present invention relates to a low-voltage, very-low-power conductance mode neuron.
BACKGROUND OF THE INVENTION
As is known, artificial intelligence systems feature neural networks for performing complex tasks, in particular, for texture analysis, morphological kernel filtering in facsimile transmission, vehicle tracking systems, pattern recognition, hardware simulation in neural CAD systems, and preprocessing in optical character recognition applications.
Neural networks employ components known as neurons, similar to the biological elements of the same name and based on the addition and subtraction of appropriately weighted inputs, and for which various mathematical formalisms have been devised. Reference is made in the present invention to binary neurons according to the McCulloch-Pitts model, which stands out for the precision and elegance of its mathematical definition, and according to which, the output may assume only two binary values “0” and “1” and operates under a discrete time assumption, with predetermined neuron thresholds and weights. Each neuron comprises a processing element with a number of synaptic input connections and one output; and input and output signal flow is considered one-way.
In the classification phase neural network parameters are fixed and the neural network executes the recognition or the analysis starting from the information contained in the topology and in the weights of the neural network.
FIG. 1
shows a symbolic representation of the McCulloch-Pitts model, in which x
1
, x
2
, . . . , x
i
are the inputs, w
1
, w
2
, . . . , w
i
the weights, and O the output. The neuron is represented by a node defining function f, which, when applied to the weighted inputs, supplies the output according to the equation:
 
O=f
(&Sgr;
w
i
*x
i
)
Typically, the f function compares the sum of the weighted products of the inputs with a threshold, and, depending on the outcome of the comparison, determines the binary value of the output.
Various solutions are known for implementing neural networks, as described for example in S. Satyanarayana et al., “A Reconfigurable VLSI Neural Network,”
IEEE Journal of Solid-State Circuits
27:1, January 1992; B. E. Boser et al., “An Analog Neural Network Processor with Programmable Topology,”
IEEE Journal of Solid-State Circuits
26:12, December 1991; and A. Kramer et al., “EEPROM Device As a Reconfigurable Analog Element for Neural Networks,” IEDM, 1989, pp. 259-262.
All known solutions, however, involve a trade-off between power consumption and precision, require a large integration area, and are complex in design. Moreover, all known prior art solutions make it necessary to choose between solutions designed for high speed but requiring high power (current mode computation) and solutions designed for low power but operating at low speed (charge computation mode).
Analog implementations of Neural Network Architectures provide a framework for computation which is more efficient than standard digital techniques for certain problems. Typically, implementations of analog neural networks have been based on the use of either current or charge as the variable of computation.
SUMMARY OF THE INVENTION
The present invention provides a new class of analog neural network circuits based on the concept of conductance-mode computation. In this class of circuits, accumulated weighted inputs are represented as conductances, and a conductance-mode neuron is used to apply nonlinearity and produce an output. In the hardware implementation of neural networks it is important to consider flexibility and power consumption in order to satisfy a wide range of applications. The present invention has been to focus on circuits which consume very little power per connection allowing for a high number of connections per neuron.
It is an object of the present invention to provide a compact, low-power neural network with a high input range that operates at high speed. It is adaptable to different (even low-voltage) supply conditions and it requires no additional interfacing to be compatible with digital environments. It may also be used in portable devices.
One advantage of the present invention is that it implements synapses by a simple circuit based on a pair of floating-gate transistors, providing both analog multiplication and weight storage with low power consumption, high density and high precision. Both the weight storage and analog multiplication are implemented concurrently in a pair of floating gate transistors.
The advantages of this class of circuits are twofold: firstly, conductance-mode computation is fast - circuits based on these principles can compute at about 5-10 MHz; secondly, because conductance-mode computation requires the minimum charge necessary to compare two conductances, its energy consumption is self-scaling depending on the difficulty of the decision to be made. The computing precision of these circuits is high. Test results on a small test structure indicate an intrinsic precision of 8-9 bits.


REFERENCES:
patent: 4956564 (1990-09-01), Holler et al.
patent: 4961002 (1990-10-01), Tam et al.
patent: 4988891 (1991-01-01), Mashiko
patent: 5004932 (1991-04-01), Nejime
patent: 5021693 (1991-06-01), Shima
patent: 5021988 (1991-06-01), Mashiko
patent: 5053638 (1991-10-01), Furutani et al.
patent: 5056037 (1991-10-01), Eberhardt et al.
patent: 5101361 (1992-03-01), Eberhardt
patent: 5146602 (1992-09-01), Holler et al.
patent: 5150450 (1992-09-01), Swenson et al.
patent: 5155377 (1992-10-01), Castro
patent: 5155802 (1992-10-01), Mueller et al.
patent: 5187680 (1993-02-01), Engeler
patent: 5202956 (1993-04-01), Mashiko
patent: 5248956 (1993-09-01), Himes et al.
patent: 5256911 (1993-10-01), Holler et al.
patent: 5258657 (1993-11-01), Shibata et al.
patent: 5268320 (1993-12-01), Holler et al.
patent: 5274746 (1993-12-01), Mashiko
patent: 5298796 (1994-03-01), Tawel
patent: 5299286 (1994-03-01), Imondi et al.
patent: 5305250 (1994-04-01), Salam et al.
patent: 5336937 (1994-08-01), Sridhar et al.
patent: 5343555 (1994-08-01), Yayla et al.
patent: 5396581 (1995-03-01), Mashiko
patent: 5422982 (1995-06-01), Pernisz
patent: 5444821 (1995-08-01), Li et al.
patent: 5475794 (1995-12-01), Mashiko
patent: 5509105 (1996-04-01), Roenker et al.
patent: 5615305 (1997-03-01), Nunally
patent: 5704014 (1997-12-01), Marotta et al.
patent: 6032140 (2000-02-01), Fabbrizio et al.
patent: 0 349 007 (1990-01-01), None
Lee et al., “Analog Floating-Gate Synapses for General-Purpose VLSI Neural Computation,” IEEE Transactions on Circuits and Systems, vol. 38, No. 6, pp. 654-658, Jun. 1991.*
White et al., “Electrically Modifiable Nonvolatile Synapses for Neural Networks,” IEEE International Symposium on Circuits and Systems, vol. 2, pp. 1213-1216, May 1989.*
Mizugaki et al., “Implementation of New Superconducting Neural Circuits using Coupled SQUIDs,” IEEE Transcriptions on Applied Superconductivity, vol. 4, No. 1, pp. 1-8, Mar. 1994.*
Kramer et al., “EEPROM Device as a Reconfigurable Analog Element for Neural Networks,” International Electron Devices Meeting, 1989. Technical Digest, pp. 259-262, Dec. 1989.*
Sze, S.M.,Physics of Semiconductor Devices, 2 ed., Wiley, 1981, p. 403.
Jain, Jaswant R. and Anil K. Jain, “Displacement Measurement and Its Application in Interframe Image Coding,”IEEE Transactions on Communications29(12):1799-1808, Dec. 1981.
Degrauwe et al., “A Micropower CMOS-Instrumentation Amplifier,”IEEE J. of Solid-State Circuits20(3):805-807, Jun. 1985.
Sage et al., “An Artificial Neural Network Integrated Circuit Based on MNOS/CCD Principles,” Proc. Conf. Neural Networks for Computing, J.S. Denker (ed.), Amer. Inst. of Physics, Snowbird, UT, 1986, pp. 381-385.
Graf et al., “A CMOS Associative Memory Chip,” Proc. IEEE First International Conference Neural Networks, M. Caudill and C. Butler (ed.), SOS Printing, San Diego, CA, 1987, pp. III-461-III-468.
Seevinck, Evert and Roelof F. Wassenaar, “A Versatile CMOS Linear Transconductor/ Square-Law Function Circuit,”IEEE J. of Solid-State Circuits22(3):366-377, Jun. 1

Affiliated with

Also associated with

Rating

Low-voltage, very-low-power conductance mode neuron does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Low-voltage, very-low-power conductance mode neuron, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Low-voltage, very-low-power conductance mode neuron will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-2511811