Data processing: artificial intelligence – Neural network – Learning task
Reexamination Certificate
1999-05-10
2003-06-17
Powell, Mark R. (Department: 2122)
Data processing: artificial intelligence
Neural network
Learning task
C706S016000, C250S369000
Reexamination Certificate
active
06581048
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention is directed to a neural network control system including, in one embodiment, a computer-implemented method and apparatus using a computer-readable medium to control a general-purpose computer to perform intelligent control.
2. Description of the Background
Science has been fascinated by the capabilities of the human mind, and many have hypothesized on the process by which mammalian brains (and human brains in particular) learn. When NSF first set up the Neuroengineering program in 1987, it was not motivated by any kind of desire to learn more about the brain for its own sake. The program was set up as an exercise in engineering, as an effort to develop more powerful information processing technology. The goal was to understand what is really required to achieve brain-like capabilities in solving real and difficult engineering problems, without imposing any constraints on the mathematics and designs except for some very general constraints related to computational feasibility. In a sense, this could be characterized as abstract, general mathematical theory; however, these designs have been subjected to very tough real-world empirical tests, in proving that they can effectively control high-speed aircraft, chemical plants, cars and so on—empirical tests which a lot of “models of learning” have never been confronted with.
More precisely, the Neuroengineering program began as an offshoot of the Lightwave Technology (LWT) program at NSF. LWT was and is one of the foremost programs in the U.S. supporting the most advanced research in optical technology. It furthers the development and use of advanced optical fibers, lasers, holography, optical interface technology, and so on, across a wide range of engineering applications—communication, sensing, computing, recording, etc. Years ago, several of the most advanced engineers in this field came to NSF and argued that this kind of technology could be used to generate computing systems far more powerful than conventional electronic computers.
The desktop computer has advanced remarkably over the computers of twenty years ago. It is called a “fourth generation” computer, and its key is its Central Processing Unit (CPU), the microchip inside which does all the real substantive computing, one instruction at a time. A decade or two ago, advanced researchers pursued a new kind of computer—the fifth generation computer, or “massively parallel processor” (MPP) or “supercomputer.” The MPP may contain hundreds or thousands of CPU chips, all working in parallel, in one single box. In theory, this permits far more computing horsepower per dollar; however, it requires a new style of computer programming, different from the one-step-at-a-time FORTRAN or C programming that most people know how to use. The U.S. government has spent many millions of dollars trying to help people learn how to use the new style of computer programming needed to exploit the power of these machines.
In the late 1980's, the optical engineering seemed to be a viable basis for developing a sixth generation of computing, as far beyond the MPP as the MPP is beyond the ordinary PC. Using lasers and holograms and such, it was believed that a thousand to a million times more computing horsepower per dollar could be produced compared to the best MPP. However, although skeptics agreed that optical computing might be able to increase computing horsepower as claimed, it would require a price. Using holograms, huge throughput can be achieved, but very simple operations are required at each pixel of the holograms. This requires replicating very simple operations performed over and over again in a stereotyped kind of way, and the program is not easily replaced like a FORTRAN program can be replaced or changed.
Carver Mead, from CalTech, then pointed out that the human brain itself uses billions and billions of very simple units—like synapses or elements of a hologram—all working in parallel. But the human brain is not a niche machine. It seems to have a fairly general range of computing capability. Thus the human brain becomes an existence proof, to show that one can indeed develop a fairly general range of capabilities, using sixth generation computing hardware. The Neuroengineering program was set up to follow through on this existence proof, by developing the designs and programs to develop those capabilities. In developing these designs, advances in neuroscience are used, but they are coupled to basic principles of control theory, statistics and operations research.
However, sometimes terminology clouds advances in one area that are applicable in another area. Some computational neuroscientists have built very precise models that look like neural nets and use little circles and boxes representing differential equations, local processing and so on. Other people use artificial neural nets to accomplish technological goals. Further other scientists, including psychologists, use yet another set of terminology. What is going on is that there are three different validation criteria. In the computational neuroscience people are asking, “Does it fit the circuit?” In connectionist cognitive science they are asking, “Does it fit the behavior?” In our neuroengineering, people are asking, “Does it work? Can it produce solutions to very challenging tasks?” But in actuality, whatever really goes on in the brain has to pass all three tests, not just one. Thus logic suggests a combination of all three validation criteria is needed.
Present models must go beyond the typical test of whether or not a model can produce an associative memory. The bottom line is that a new combination of mathematics is needed.
Most of the engineering applications of artificial neural nets today are applications of a very simple idea called supervised learning, shown in FIG.
2
. Supervised learning is a very simple idea: some inputs (X), which are really independent variables, are plugged into a neural network, and a desired response or some target (Y) is output. Some weights in the network, similar to synapse strengths, are adapted in such a way that the actual outputs match the desired outputs, across some range of examples. If properly trained, good results are obtained in the future, when new data is applied to the network. These systems do have practical applications, but they do not explain all the functioning of the brain. To make things work in engineering a few components have to be added, above and beyond cognition. A robot that does not move is not a very useful robot. But even supervised learning by itself does have its uses.
For historical reasons, a majority of ANN applications today are based on the old McCulloch-Pitts model of the neuron, shown in FIG.
3
. According to this model, the voltage in the cell membrane (“net”) is just a weighted sum of the inputs to the cell. The purpose of learning is simply to adjust these weights or synapse strengths. The output of the cell is a simple function (“s”) of the voltage, a function whose graph is S-shaped or “sigmoidal.” (For example, most people now use the hyperbolic tangent function, tanh.) Those ANN applications which are not based on the McCulloch-Pitts neuron are usually based on neuron models which are even simpler, such as radial basis functions (Gaussians) or “CMAC” (as described in D. White and D. Sofge, eds., “Handbook of Intelligent Control,” published by Van Nostrand, 1992; and W. T. Miller, R. Sutton & P. Werbos (eds), “Neural Networks for Control,” published by MIT Press, 1990).
Although in most applications today, the McCulloch-Pitts neurons are linked together to form a “three-layered” structure, as shown in
FIG. 4
, where the first (bottom) layer is really just the set of inputs to the network, it is known that the brain is not so limited. But even this simple structure has a lot of value in engineering. Further, there are some other concepts that have arisen based on the study of neural networks: (1) all neural networks approximate “nice” fu
Powell Mark R.
Starks, Jr. Wilbert L.
LandOfFree
3-brain architecture for an intelligent decision and control... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with 3-brain architecture for an intelligent decision and control..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and 3-brain architecture for an intelligent decision and control... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3122986