Dynamic memory processor

Details Dynamic memory processor Dynamic memory processor

2000-01-05

2003-05-06

Follansbee, John (Department: 2121)

Data processing: artificial intelligence

Neural network

C706S018000, C706S020000, C706S027000

Reexamination Certificate

active

06560582

ABSTRACT:

BACKGROUND OF THE INVENTION
(1) Field of the Invention
The present invention relates generally to the field of electronic neural networks, and more particularly to a new architecture for neural networks having a plurality of hidden layers, or multi-layer neural networks, and further to new methodologies for providing supervised and unsupervised training of neural networks constructed according to the new architecture.
(2) Description of the Prior Art
Electronic neural networks have been developed to rapidly identify patterns in certain types of input data, or to accurately classify the input patterns into one of a plurality of predetermined classifications. For example, neural networks have been developed which can recognize and identify patterns, such as the identification of hand-written alphanumeric characters, in response to input data constituting the pattern of on/off picture elements, or “pixels,” representing the images of the characters to be identified. In such a neural network, the pixel pattern is represented by, for example, electrical signals coupled to a plurality of input terminals, which, in turn, are connected to a number of processing nodes, or neurons, each of which is associated with one of the alphanumeric characters which the neural network can identify. The input signals from the input terminals are coupled to the processing nodes through certain weighting functions, and each processing node generates an output signal which represents a value that is a non-linear function of the pattern of weighted input signals applied thereto. Based on the values of the weighted pattern of input signals from the input terminals, if the input signals represent a character which can be identified by the neural network, one of the processing nodes which is associated with that character will generate a positive output signal, and the others will not. On the other hand, if the input signals do not represent a character which can be identified by the neural network, none of the processing nodes will generate a positive output signal. Neural networks have been developed which can perform similar pattern recognition in a number of diverse areas.
The particular patterns which the neural network can identify depend on the weighting functions and the particular connections of the input terminals to the processing nodes, or elements. As an example, the weighting functions in the above-described character recognition neural network essentially will represent the pixel patterns which define each particular character. Typically, each processing node will perform a summation operation in connection with the weight values, also referred to as connection values or weighting values, representing the weighted input signals provided thereto, to generate a sum that represents the likelihood that the character to be identified is the character associated with that processing node. The processing node then applies the non-linear function to that sum to generate a positive output signal if the sum is, for example, above a predetermined threshold value. The non-linear functions which the processing nodes may use in connection with the sum of weighted input signals are generally conventional functions, such as step functions, threshold functions, or sigmoids. In all cases the output signal from the processing node will approach the same positive output signal asymptotically.
Before a neural network can be useful, the weighting functions for a set of the respective input signals must be established. In special cases, the weighting functions can be established a priori. Normally, however, a neural network goes through a training phase, in which input signals representing a number of training patterns for the types of items to be classified (e.g., the pixel patterns of the various hand-written characters in the character-recognition example) are applied to the input terminals, and the output signals from the processing nodes are tested. Based on the pattern of output signals from the processing nodes for each training example, the weighting functions are adjusted over a number of trials. Once trained, a neural network can generally accurately recognize patterns during an operational phase. The degree of success is based in part on the number of training patterns applied to the neural network during the training stage and the degree of dissimilarity between patterns to be identified. Such a neural network can also typically identify patterns which are similar to the training patterns.
One of the problems with conventional neural network architectures as described above is that the training methodology, generally known as the “back-propagation” method, is often extremely slow in a number of important applications. Also, under the back-propagation method, the neural network may provide erroneous results which may require restarting the training. In addition, even after a neural network has been through a training phase, confidence that the best training has been accomplished may sometimes be poor. If a new classification is to be added to a trained neural network, the complete neural network must be retrained. Further, the weighting functions generated during the training phase often cannot be interpreted in ways that readily provide understanding of what they particularly represent.
SUMMARY OF THE INVENTION
Accordingly, it is an object of the present invention to provide a new and improved neural network architecture for use in pattern recognition in which the weighting functions may be determined a priori.
Another object of the present invention is to provide a neural network architecture which can be trained with a single application of an input data set.
A further object of the present invention is to provide a neural network architecture which can be used in time varying pattern recognition.
Other objects and advantages of the present invention will become more obvious hereinafter in the specification and drawings.
In accordance with the present invention, a new neural network architecture, referred to hereinafter as a dynamic memory processor, is provided. The dynamic memory processor receives inputs from a sensor and provides low dimensional pattern recognition or classification identifiers for the inputs. The dynamic memory processor provides pattern recognition for time variant inputs, such as sound signal inputs. The dynamic memory processor is part of a new neural network technology that is constructed rather then trained. Since the words “neural networks” often connote a totally trainable neural network, a constructed neural network is a connectionist neural network device that is assembled using common neural network components to perform a specific process. The constructed neural network assembly is analogous to the construction of an electronic assembly using resistors, transistors, integrated circuits and other simple electronic parts. A constructed neural network is fabricated using common neural network components such as processing elements (neurons), output functions, gain elements, neural network connections of certain types or of specific values and other artificial neural network parts. As in electronics, the design goal and the laws of nature such as mathematics, physics, chemistry, mechanics, and “rules of thumb” are used to govern the assembly and architecture of a constructed neural network. A constructed neural network, which is assembled for a specific process without the use of training, can be considered equivalent to a trained neural network having accomplished an output error of zero after an infinite training sequence. Although there are some existing connective circuits that meet the design criteria of a constructed neural network, the term “constructed neural network” is used herein to differentiate this new neural technology which does not require training from the common neural network technology requiring training. A constructed neural network can consist of a neural network module, such as a neural director, a neural sensor, or a vector decoupler, and may con

Affiliated with

Also associated with

Rating

Dynamic memory processor does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Dynamic memory processor, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Dynamic memory processor will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-3032604