Data processing: artificial intelligence – Neural network – Structure
Reexamination Certificate
1998-11-30
2001-07-31
Davis, George B. (Department: 2122)
Data processing: artificial intelligence
Neural network
Structure
C706S038000
Reexamination Certificate
active
06269354
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of Invention
The invention pertains to the fields of pattern recognition, scene segmentation, machine vision and machine speech recognition, and neuromorphic engineering. The invention constitutes a group of elementary circuits composed of interconnected e-cells (the subject of a related application) that themselves can be assembled into a new class of systems to implement perceptual and cognitive functions in various sensory and cognitive domains.
2. Discussion of Related Art
The history of neurologically inspired computing technology goes back at least several decades. The field lay relatively quiescent until the mid 1980s when the popularization of the multi-layer “artificial neural net” (ANN), usually emulated in software, overcame a decade and a half of skepticism about the technology largely engendered by an influential book “Perceptrons” by Marvin Minsky and Seymour Papaert. The multilayer ANN, particularly ones trained by various algorithms of backward propagation of an error signal, proved capable of a wide range of classification tasks. Though the differences between ANNs and biological neural circuitry was evident, the lack of an alternative computational hypothesis for the biological circuitry at first attracted some in the neuroscience community to adopt the ANN in its various forms as a model of neural computation. The limitations of the ANN, however, became apparent in both technological application and as a computational model for the brain. The ANN has become an accepted part of the computational repertoire for solving a certain classes of classification problems and problems involving capturing of complex functions from a large set of training data.
However, the ANN on its own has not provided the basis for solutions to the more complex problems of vision, speech recognition and other sensory and cognitive computation. Problems of segmentation, recognition of signals under various transformations, learning from single or very limited presentations of data, regularity extract, and many other real world recognition problem have to date eluded solution by ANN or any other neurally inspired techniques.
Experimental neuroscience has revealed that cortical and other neural architecture is far more complicated than any ANN. Realistic simulations of large neurons such as pyramidal or Purkinje cells suggest that individual neurons are capable of significant computation on their own, which must be the basis of the computations performed by circuits containing myriads of such cells. Experiment has also determined that connectivity between neurons is highly specific and becomes more so during the initial learning by organisms. Neurons themselves are specialized, not uniform. Strong evidence exists that synchronized oscillations across the cortices play a role in recognition. And perhaps most important is the fact that backward or descending signal paths are at least as numerous as the forward or ascending signal paths from sensory organs to progressively “higher” cortices.
BRIEF DESCRIPTION OF THE INVENTION
The invention consists of a number of e-circuits which are expansions of the basic memory e-circuit (subject of another application) to build a general purpose recognition e-circuit capable of learning presented patterns and later recognizing them, or subsets of them, when presented anywhere in an input field in the presence of other percepts and distractors. The invention implements translation invariant recognition, recognition under degradation due to noise or occlusion, input field segmentation, and attentional shift between concurrently presented patterns corresponding to memorized percepts.
In addition e-circuitry is added which permits patterns to be learned when presented anywhere in an input field, and compacted if broken up across the input field.
The basic memory e-circuit (subject of associated application E-Cell (Equivalent Cell) and The Basic Circuit Modules of E-Circuits: E-Cell Pair Totem, The Basic Memory Circuit and Association Extension) is extended by one standard e-cell pair stage, stage r. Stage r is richly interconnected to stage b, both forward and backward, allows features present anywhere across stage r to find a target region of stage m whose weights match its pattern of activation. The resulting e-circuit is capable of recognizing patterns in any translation, without incurring a time cost proportional to the space searched for a match, but also suffers from a complete of lack of order selectivity. As a result, anagrams of memorized percepts are recognized without distinction from the originally memorized pattern. In most applications this is not acceptable and must be remedied. The backward path activation permits e-circuitry to be added to the three-stage e-circuit which corrects the lack of order selectivity. These additional circuit enhancements are termed r-match and r-match diagonalization. With a relatively small addition of e-circuitry, the order enforcement by the recognition e-circuit can be made arbitrarily strict without imposing a time penalty on the recognition and without losing the translation tolerance (or invariance) of the e-circuit.
When multiple patterns corresponding to memorized percepts are presented in a single input field, it is useful in real applications to be able to split up the input field (segmentation) along the boundaries of the recognized percepts and identify each recognized percept with its local in the input field. This is the basis of sentence parsing and visual scene recognition. An attention focus enhancement to the recognition e-circuit, also driven from the recognition signal propagated along the backward path to stage r allow the circuit to shift attention between concurrently presented patterns in the input field.
The invention also consists of a method of application of the r-b-m e-circuit including the enhancements described in the associated applications to the problem of machine vision. The resulting recognition engine is capable of learning percept images and then recognizing and segmenting those percepts when they occur in a compound image even under transformations of translation, rotation and scaling as well as some perspective transforms. The recognition time for a given percept is independent of the size of the input field in which that percept occurs. Typical recognition latencies are three to six oscillatory cycles, and the switch of attention from one percept to another anywhere in the input field takes under a dozen oscillatory cycles. The real-time length of an oscillatory cycle is dependent on the implementation medium: for analog VLSI implementations, oscillatory frequencies in the megahertz range are practical.
The recognition characteristics of the r-b-m e-circuit are applied to two-dimensional patterns by suitable mapping between a two dimensional field of feature detectors and a locally one-dimensional extent of the input stage r. The characteristics of the feature detector field which provides input are independent of the invention described here. The circuit will store and recognize percepts characterized in whatever feature sets provide the input to the circuit.
The input stage r of the e-circuit samples the feature detector field in such a manner that areas of the image are mapped via an r-match/r-intersect “diagonal” onto stage b. Each “diagonal” is a surjective mapping of some subset of stage r e-cell pairs onto the full set of stage b e-cell pairs. All mapped subset of stage r e-cell pairs represent contiguous areas of the input feature field corresponding to translations and, if necessary, rotations, scalings and perspective transforms of the percept field. The ability of the diagonalizing r-b-m e-circuit to select the diagonal which best maps an active subset of stage r to some percept stored in stage m in one or a very few cycles of e-cell activity allow percepts present in the input image to be rapidly identified even when translated, rotated, etc. Furthermore, the ability of the r-b-m e-circuit equipped with attention focus cir
Davis George B.
Fliesler Dubb Meyer & Lovejoy LLP
LandOfFree
General purpose recognition e-circuits capable of... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with General purpose recognition e-circuits capable of..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and General purpose recognition e-circuits capable of... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2565761