Data processing: artificial intelligence – Neural network – Structure
Reexamination Certificate
2001-03-20
2004-06-22
Starks, Jr., Wilbert L. (Department: 2121)
Data processing: artificial intelligence
Neural network
Structure
C706S038000, C706S033000
Reexamination Certificate
active
06754645
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates to a neural network, particularly to a VLSI implementation of neural networks. It shortens the convergence time and eliminates the influence of the initial value by using the GA features, thereby achieving the conservation of hardware resources.
2. Description of the Related Art
Because of its independence on specific model of signal processing, artificial neural networks (ANN) is now widely used in the fields of image processing and pattern recognition. Refer to
FIG. 1. A
multilayer neural network typically comprising an input layer, an output layer, and a hidden layer. The input layer having a row of cells
1
, . . . ,
2
or neuron-like units (not shown) serves as an interconnect function, externally connecting a plurality of inputs x
1
, . . . , x
n
to the network, wherein each input is coupled to a respective input cell. The hidden layer having a plurality of neuron cells
3
, . . . ,
4
, which is the most important part in the network and has multiple inputs and one output, is responsible for performing the specific functions of the network, wherein the output of each input cell in the input layer is coupled to each of a plurality of neuron cells in the hidden layer. For example, the specific functions can be that each neuron cell multiplies the inputs x
1
, . . . , x
n
by a given weight w
1
, w
2
, . . . , w
3
, w
4
and adds the products together, as shown in
3
and
4
in the hidden layer. The output layer having at least one cell
5
or neuron-like unit (not shown) serves as the output collection of a plurality of neuron cells
3
, . . . ,
4
in the hidden layer. For such a network, although now the majorities of neural systems are realized by software simulations, only hardware implementations can fully utilize its advantages of parallel processing and error tolerance. Among the existing 3 methods of VLSI implementation for neural systems, analog circuit has the merits of small chip area, fast speed and low power consumption, but is susceptible to process parameters and interface noises. Digital circuit has the merits of high precision and robustness, but it is area and power consuming. The digit-analog hybrid method—pulse stream technique combines both the merits of analog and digital methods. It uses digital signal to control analog multiplication, so the area of synapse analog multiplier is small and the neuron digital state signal is immune to noise. Among the various kinds of pulse modulation techniques, pulse width modulation (PWM) is the easiest for circuit implementation, so the PWM VLSI neural network is widely studied.
Although the first proposed structure of 3-transistor synapse multiplier is simple, it has low precision and small linearity range because of the switching noise and working states of the transistors. Furthermore, it uses external non-linear voltages for activation function, which is not suitable for VLSI implementation because of the limited interface. A low gain 2-transistor amplifier is adopted for activation function, but its transfer characteristic is quite different from the ideal sigmoid function, so it is not suitable for ANNs which use Back Propagation (BP) learning algorithm. Although the transfer characteristic of asynchronous neuron proposed is very similar to ideal sigmoid function, it needs complex BiCMOS process. Another novel neuron circuit can easily realize both sigmoid function and its derivative, but its current input is not suitable for voltage-mode sigmoid circuit.
SUMMARY OF THE INVENTION
Therefore, an object of the invention is to provide a new building blocks for PWM VLSI neural network, including a simple precise synapse multiplier with large linearity range and a voltages-pulse converter with high precision.
Another object of the invention is to provide a CMOS voltage-mode sigmoid circuit with adjustable gain, whose transfer function is similar to ideal sigmoid function.
To realize the above and other objects, the invention provides a voltage-mode pulse width modulation (PWM) VLSI implementation of neural networks, including a simple precise synapse multiplier with large linearity range and a voltages-pulse converter with high precision. Further, A CMOS voltage-mode sigmoid circuit with adjustable gain is designed, whose transfer function is similar to ideal sigmoid function, thereby constructing a 2-2-1 feedforward PWM ANN.
REFERENCES:
patent: 4893255 (1990-01-01), Tomlinson, Jr.
patent: 5165009 (1992-11-01), Watanabe et al.
patent: 5165010 (1992-11-01), Masuda et al.
patent: 5426757 (1995-06-01), Watanabe et al.
patent: 5594916 (1997-01-01), Watanabe et al.
patent: 5671337 (1997-09-01), Yoshihara
patent: 5696883 (1997-12-01), Arima
patent: 5875347 (1999-02-01), Watanabe et al.
patent: 2002/0032670 (2002-03-01), Watanabe et al.
Dayhoff, J., “Pattern mapping in Pulse Transmission Neural Networks”, Proceedings of the conference on analysis of neural network applications, 1991, pp. 146-159.*
Gu et al., “A Novel Current-Mode Fuzzy Processor for Pattern Recognition”, 4th International Conference on Signal Processing Oct. 1998, vol. 1, pp. 541-544.*
Gu et al., “A Novel Switched-Current Sorter Based on Magnitude”, 5th International Conference on Solid-State and Integrated Circuit Technology, Oct. 1998, pp. 393-396.*
Chen et al., “CMOS PWM VLSI Implementation of Neural Network”, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, Jul. 2000, vol. 3, pp. 485-488.*
Chen et al., “Building Blocks for PWM VLSI Network”, 5th International Conference on Signal Processing, Aug. 2000 vol. 1, pp. 563-566.*
Lazzaro, J., “Low-Power Silicon Spiking Neurons and Axons”, IEEE International Symposium on Circuits and Systems, 1992.*
Astaras et al., “Pulse-Based Circuits and Methods from Probabilistic Neural Computing”, Proceedings of the 7thInternational Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems, Apr. 1999, pp. 96-102.*
Reyneri, L.M., “A Performance Analysis of Pulse Stream Neural and Fuzzy Computing Systems”, IEEE Transactions on Circuit and Systems II: Analog and Digital Signal Processing, Oct. 1995, Vol 42, Iss 10, pp. 642-660.
Lu Chun
Shi Bingxue
Booker Kelvin
Ladas & Parry
Starks, Jr. Wilbert L.
Winbond Electronics Corp.
LandOfFree
Voltage-mode pulse width modulation VLSI implementation of... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Voltage-mode pulse width modulation VLSI implementation of..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Voltage-mode pulse width modulation VLSI implementation of... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3333774