Data processing: artificial intelligence – Neural network – Structure
Reexamination Certificate
2001-10-01
2004-12-07
Patel, Ramesh (Department: 2121)
Data processing: artificial intelligence
Neural network
Structure
C706S033000, C365S200000
Reexamination Certificate
active
06829598
ABSTRACT:
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT
Not Applicable
Reference to a “Microfiche Appendix”
Not Applicable
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to modeling the function of a neural synapse in hardware thus allowing for building of neural networks or alike learning and adaptive systems using low-power conventional MOSFETs integrated on-chip with other analog or digital application circuits.
2. Description of the Related and Prior Art
In spite of the immense advance in the development of the modem VLSI technology, digital circuits or systems reach the critical point from its computational speed and integration. In implementing real-time digital signal processing applications, the conventional VLSI technology has many limitations with regard to chip area and its speed, due to complexity of both hardware and software processing algorithms. On the other hand, artificial neural networks offer great potential in many applications. This poses the need of a simple, fast and efficient solution for implementation of such neural networks in a conventional CMOS fabrication technology along with the rest of the integrated circuit application. Such characteristics can be most often satisfied by the utilization of simple analog signal processing elements, in particular MOSFETs.
Prior efforts for single-MOSFET synapse circuits largely include use of floating gate MOSFET devices (for example: U.S. Pat. No. 6,023,422; U.S. Pat. No. 5,986,927; U.S. Pat. No. 5,914,894; U.S. Pat. No. 5,864,242; U.S. Pat. No. 5,825,063; U.S. Pat. No. 5,818,081; U.S. Pat. No. 5,627,392; U.S. Pat. No. 5,621,336; U.S. Pat. No. 5,457,771; U.S. Pat. No. 5,336,936; U.S. Pat. No. 5,331,215; U.S. Pat. No. 5,253,196; U.S. Pat. No. 5,204,549; U.S. Pat. No. 5,027,171) which utilize various structures and methods to store/remove electrical charges on/from a floating gate or node through insulator by tunneling (Fowler-Nordheim) or hot-electron-injection. Beside the many advantages that those structures offer, they rely on conditions that are hard to meet in a conventional CMOS fabrication process. For example, at the voltages and oxide thicknesses used in conventional silicon MOS processing, the tunneling probability is exceedingly small, likewise hot-electron-injection requires much higher electric fields and supply voltages than normal. The latter is in direct conflict with the tendency of lowering supply voltages with the constant decrease of the device geometries and increase of the scale of integration.
Other related prior art includes analog multipliers build by coupled current mirrors (U.S. Pat. No. 5,914,868), four-quadrant square-law circuits (U.S. Pat. No. 5,864,255) or pair of MOSFETs and a symmetrical voltage source (U.S. Pat. No. 5,254,889) all capable of removing non-linear current component in one form or another to improve multiplier linearity. Some solutions utilize voltage-current conversion circuits employing differential pair of MOS transistors (U.S. Pat. No. 5,704,014) or current controlled current conveyor circuits (U.S. Pat. No. 5,206,541). There are also solutions to use multiple-bit-split MOSFET gates (U.S. Pat. No. 5,442,209). It should be mentioned that this list is not in any way comprehensive or inclusive and does not represent the complete scope and class of search for prior art performed by the inventor. For sake of briefness, herein will not be mentioned quite innovative solutions for building artificial neural networks as, for example, “Information processing device capable of optically writing synapse strength matrix” (U.S. Pat. No. 5,546,504) in which a matrix of plurality of neurons is realized through a combination of a light-emitting and light-receiving functions of two overlapping molecular films. The above-related art is mentioned in order to point out that most of the solutions require special integrated circuit technology, show performance dependencies as to the quality or the technology parameters of the used fabrication process or require special ambient (for UV floating-gate programming for example). Therefore, in general practice, those solutions do not allow for easy on-chip integration with other integrated circuit applications implemented on most widely used conventional CMOS processes.
Naturally, it is desired to provide a simple single-transistor synapse for modeling neural synaptic connection function and for building artificial neural networks in VLSI integrated circuits, which offers following advantages and features:
1. use of a single, conventional MOSFET to model a neural synapse function;
2. low power consumption;
3. independence of the model quality with regard to quality of the CMOS fabrication process;
4. simplicity and feasibility to dense integration;
5. preferred conventional CMOS implementation which allows for integration with conventional analog and digital IC applications;
6. current signal input and output allowing for straightforward and easy multi-layer neural network interconnection, simple synapse-output summing, avoidance of parasitic capacitive effects on both input and output transient signals therefore allowing for faster processing in real-time applications;
This and other features of the invention will be apparent to a person of ordinary skill in the art from the description of the invention contained herein.
Terms and Definitions
Specification description of the claimed invention and disclosure will be facilitated by the definition of the meaning of the following terms. Term definitions given herein are intended to help in limiting redundancy and unduly multiplication of terms in the disclosure and should be used to better comprehend the spirit and scope of the invention. The definitions given are not intended to limit and define the scope of the appended claims rather than to provide antecedent support of a claim in cases where such support can be more easily found by the interpretation of the terms given herein and used in the disclosure, or in cases where these definitions help avoid inconsistencies between claim and specification disclosure or prior art, should such arise.
Neural synapse—although ‘neural’ refers to a biological synaptic connection formed between a dendrite tree and axon terminal of a neuron (most often cerebral pyramidal cell), herein it should be most often interpreted as artificial synapse model, mathematical or functional abstract of its biological counterpart. Synapse, neural synapse or synaptic connection will most often be used interchangeably in the application.
Synapse strength or weight—the quantity that can be used to show the degree of proportion between the quantity used to represent pre-synaptic activity (input stimulus) and the quantity representing post-synaptic activity (weighted input stimulus).
Neural synapse function—although biological synapse function comprises functions of excitation or inhibition to effectively strengthen or weaken the transmission of a pre-synaptic signal to a post-synaptic signal and also learning functions (to keep or adjust its strength with respect to synaptic activity), herein, a synapse function should be interpreted as the multiplication function between the pre-synaptic signal quantity and synapse strength (or weight) quantity employed in the computational model most often used to describe the synapse.
Neuron—most often should be interpreted as functional abstract model of its biological counterpart. The latter model comprises of a linear combiner producing a sum of weighted input stimuli, further processed by a non-linear activation function, of sigmoidal or hard-limiting type, applied to limit the amplitude of the neuron output activity potential.
Neural network—a collection of interconnected neurons where a neuron input is either connected to previous layer neuron's output or receives input stimulus from an input node. Since the type or actual connectivity structure is immaterial to the present invention no further definition of neural network types and connectivity patterns will be given herein.
SUMMAR
Brady W. James
Holmes Michael B.
Patel Ramesh
Swayze, Jr. W. Daniel
Telecky , Jr. Frederick J.
LandOfFree
Method and apparatus for modeling a neural synapse function... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for modeling a neural synapse function..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for modeling a neural synapse function... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3307665