Data processing: artificial intelligence – Neural network – Structure
Reexamination Certificate
1999-12-15
2001-11-27
Davis, George B. (Department: 2122)
Data processing: artificial intelligence
Neural network
Structure
C706S019000
Reexamination Certificate
active
06324532
ABSTRACT:
The present invention relates generally to the field of neural information processing and, more particularly, to a hierarchical apparatus and concomitant method for learning and integrating features from multiple resolutions for detecting and/or classifying objects, especially larger objects.
BACKGROUND OF THE INVENTION
Neural network modeling has been developed to solve problems ranging from natural language understanding to visual processing. A neural network is a computational model composed of neurons (also known as nodes, units or perceptrons) and connections between the nodes. The strength of each connection is expressed by a numerical value called a weight, which can be modified. The activation of a given node is based on the activations of the nodes that have connections directed at that node and the weights on those connections.
In contrast to conventional computers, which are programmed to perform specific tasks, most neural networks do not follow rigidly programmed rules and are generally taught or trained. Generally, feed-forward neural network can be implemented as functions y(f,w) of a vector f of inputs and a weight or parameter vector w. The weight vector is modified such that the neural network optimally estimates some quantity that depends on f. The process of adjusting w is commonly referred to as training, where the methods for training are referred to as training algorithms. Most neural network trainings involve the use of an error function. The weight vector is adjusted so as to minimize the sum of average of the error function on a set of training samples. A penalty term is generally applied to the error to restrict the weight vector in some manner that is thought desirable. Given the resulting objective function, various training methods are used to minimized it or involve the use of some form of gradient descent.
For instance, in image analysis a digital photographic image can be introduced to a neural network for identification, and it will activate the relevant nodes for producing the correct answer based on its training. Connections between individual nodes are “strengthened” (resistance turned down) when a task is performed correctly and “weakened” (resistance turned up) if performed incorrectly. In this manner a neural network is trained and provides more accurate output with each repetition of a task.
The field of image analysis is well-suited for computer-assisted search using neural network. Generally, images contain a vast quantity of information where only a small fraction of the information is relevant to a given task. The process of identifying the relevant fraction from the vast quantity of information often challenges the capabilities of powerful computers. Although neural networks have demonstrated its flexibility as pattern-recognition apparatus for detecting relevant information from images, they scale poorly with the size of the images. As the size of the image and neural network increases, the computational expense and training time may become prohibitive for many applications.
For example, radiologists are faced with the difficult task of analyzing large quantities of mammograms to detect subtle cues to breast cancer which may include the detection of microcalcifications. A difficult problem is the detection of small target objects in large images. The problem is challenging because searching a large image is computationally expensive and small targets on the order of a few pixels in size have relatively few distinctive features which enable them to be identified from “non-targets”.
A second problem is the need for using real data (training samples) to train a neural network to detect and classify objects. Such real data will almost inevitably contain errors, thereby distorting the conditional probability that an input vector came from an instance of the class that a neural network is designed to detect or from a specific position on the image.
Therefore, a need exists in the art for a method and apparatus for automatically learning and integrating features from multiple resolutions for detecting and/or classifying objects. Additionally, a need exists in the art for a supervised learning method that addresses errors in the training data.
SUMMARY OF THE INVENTION
A signal processing apparatus and concomitant method for learning and integrating features from multiple resolutions for detecting and/or classifying objects are presented. Specifically, the signal processing apparatus comprises a hierarchical pyramid of neural networks (HPNN). In one embodiment, the HPNN has a “fine-to-coarse” structure. In an alternative embodiment, the HPNN has a network architecture is a combination of the “fine-to-coarse” HPNN and the “coarse-to-fine” HPNN. These HPNNs are robust in detecting target objects that may be relatively large in size when compared to the size of a region of interest (ROI).
An objective function and various associated regularizer embodiments are also presented to train the neural networks to detect sub-patterns of features of some class of objects. These training methods are useful in training a plurality of neural networks that are organized into a hierarchical relationship such as the HPNN. Additionally, these training methods are useful in training a plurality of neural networks that have multiple outputs.
REFERENCES:
patent: 4858147 (1989-08-01), Conwell
patent: 5048097 (1991-09-01), Gaborski et al.
patent: 5052043 (1991-09-01), Gaborski
patent: 5155801 (1992-10-01), Lincoln
patent: 5179441 (1993-01-01), Anderson et al.
patent: 5222195 (1993-06-01), Alkon et al.
patent: 5260871 (1993-11-01), Goldberg
patent: 5301681 (1994-04-01), DeBan et al.
patent: 5331550 (1994-07-01), Stafford et al.
patent: 5345539 (1994-09-01), Webb
patent: 5384895 (1995-01-01), Rogers et al.
patent: 5444796 (1995-08-01), Omstein
patent: 5463548 (1995-10-01), Asada et al.
patent: 5479576 (1995-12-01), Watanabe et al.
patent: 5491627 (1996-02-01), Zhang et al.
patent: 5554273 (1996-09-01), Demmin et al.
patent: 5572596 (1996-11-01), Wildes et al.
patent: 5729631 (1998-03-01), Wober et al.
patent: 5737716 (1998-04-01), Bergstrom et al.
patent: 5740326 (1998-04-01), Boulet et al.
patent: 6021369 (2000-02-01), Kamihira et al.
patent: WO 97/29437 (1997-08-01), None
Manuela S. Pereira et al. Hierarchical Neural Network For Multiresolution Image Analysis IEEE 1996, pp. 261-264.
Siegfried Bos et al. Using Weight Decay to Optimize the Generalization Ability of a Perceptron , IEEE 1996, pp. 241-246.
D. J. Burr et al. Hierarchical Recurrent Networks For Learning Musical Structure , IEEE 1993, pp. 216-225.
D.H. Ballard, “Modular Learning in Neural Networks” Proc. AAAI-87 Sixth National Conference on Artificial Intelligence, vol. 1 pp. 279-284.
M.R. Evans et al. “A Multi-Resolution Neural Network Classifier for Machine Vision” IEEE International Conference on Neural Networks, pp. 2594-2599, 1991.
T. Fechner and R. Tanger “A Hybrid Neural Network Architecture for Automatic Object Recognition” Neural Networks for Signal Processing IV, pp. 187-194.
C-H Chen et al. “Analysis of Decision Boundaries Generated by Constructive Neural Network Learning Algorithms” World Congress on Neural Networks '95, vol. 1, pp. 628-635.
D. S Chen and R. C. Jain “A Robust Back Propagation Learning Algorithm for Function Approximation” IEEE Trans. on Neural Networks, vol. 5(3), pp. 467-479.
Spence et al. “Neural Network/Pyramid Architectures that Learn Target Context”, Image Understanding Workshop, (DARPA) Defense Advance Research Project Agency, 1994.
P. Sajda, C. Spence, J. Pearson, “A Hierarchical Neural Network Architecture that Learns Target Context: Applications to Digital Mammography”, International Conference on Image Processing (ICIP), Oct. 1995.
C. Spence, J. Pearson, J. Bergen, “Coarse-to-Fine Image Search Using Neural Networks” Advances in Neural Information Processing Systems 7, The MIT Press, copyright 1995.
P. Anandan, Peter J. Burt, John C. Pearson, Clay D. Spence, “Hybrid Pyramid/Neural Network Object Recognition” Applied Imagery Pattern Recognition (AIPR), 1993.
C
Sajda Paul
Spence Clay Douglas
Burke William J.
Davis George B.
Sarnoff Corporation
LandOfFree
Method and apparatus for training a neural network to detect... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for training a neural network to detect..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for training a neural network to detect... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2610720