Data processing: generic control systems or specific application – Generic control system – apparatus or process – Optimization or adaptive control
Reexamination Certificate
1998-10-19
2001-04-10
Grant, William (Department: 2786)
Data processing: generic control systems or specific application
Generic control system, apparatus or process
Optimization or adaptive control
C700S029000, C700S048000, C700S050000
Reexamination Certificate
active
06216048
ABSTRACT:
TECHNICAL FIELD OF THE INVENTION
The present invention pertains in general to neural networks, and more particularly, to analyzing the operation of a neural network as a function of the sensitivity of input parameters on the neural network.
BACKGROUND OF THE INVENTION
Neural networks have been utilized in an ever increasing manner to predict system operation in the future such that adjustments to the system can be anticipated and also to provide control inputs to a manufacturing control system. These networks provide a non-linear representation of a plant, which non-linear representation was learned through the use of historical training data. Once the system is running, it is often desirable to change any of the operating parameters of the system through the use of either the control operation of the neural network or to determine how changes in the inputs to the neural network will affect the predicted output. These changes are often constrained by physical limitations of the plant, or by user-defined constraints supplied to achieve desired behavior. The present invention provides a novel mechanism for achieving such desired behavior while simultaneously satisfying constraints.
When utilizing neural networks as predictive tools, a system designer may be locked into the overall operating system. For example, control networks typically receive a desired output and then generate control inputs to force the control inputs to a state that will yield the desired input by minimizing the error value between a predictive output and a desired output. These control networks provide as an output a control input to the system, which then responds accordingly. However, the way in which the input is applied is not controlled. For example, a user may change the desired output to increase impurity concentration for a given process. This could entail changing the flowrate of two valves, changing the heater control, etc. Unfortunately, the operation of the plant during this change to achieve the desired output is unknown, and it is not until the entire system has settled down and the desired output has been reached that the system is operated as desired.
In order to alter the inputs to a plant to achieve a desired effect in accordance with the suggested control input changes from a control system, it is necessary for the operator and/or the control system to determine how much each input value should change and also in what the sequence of change should be. One factor that complicates the control strategy that is implemented is the sensitivity of the output on each of the input values. For example, the outputs in some plants are very sensitive to certain input values and exhibit almost no change as a function of changes in other input values. However, this fact is not known until the predictive network that represents the plant is completely trained on all input values. Therefore, the network requires all input values in order to operate. Of course, the ones of the input values that cause no change in the output can be clamped to zero, if these input variables can be discriminated, and then the network run to define or to predict new and updated input values. Therefore, in order to eliminate the least sensitive input values, it is first necessary to determine which of these input values causes little or no change in the output and then eliminate these values from the input. One problem that exists in determining the sensitivity of the input variables on the network, and subsequently eliminating the input variables as inputs to the network, is whether the dependence of the outputs on the input variables is linear or non-linear. Further, it is necessary to determine whether the output is sensitive to a combination of input variables as opposed to being sensitive only to individual input variables. For example, a given input variable in isolation may cause no change in the output. However, this input variable in combination with a change in another input variable may cause a large change in the output variable. Therefore, non-linear relationships between the input variable and the output variable must be discerned in order to accurately determine the sensitivity of the output variables on input variables.
SUMMARY OF THE INVENTION
The present invention disclosed and claimed herein comprises a network training system for reducing the number of inputs in a non-linear predictive network having stored therein a representation of a plant. A dataset is provided consisting of training patterns representing the input variables and measured output variables associated with the network, with a given pattern having input values associated with the input variables and corresponding output values associated with the measured output variables. A sensitivity processor is operable to determine the sensitivities of each of the output variables as a function of each of the input variables. The sensitivity is the effect of each of the input variables on a learned representation of the output variables, which learned representation is learned over all of the input variables. This allows the combined effects of all other input variables to be taken into account. Once the sensitivity is determined, it is compared with predetermined criteria and select ones of the input variables chosen from the dataset. A network model is then trained only on the select input variables and the associated portion of the training patterns, with the model receiving only the selected input variables. The model then provides a predictive output for the received ones of the selected input variables. The network model is trained in accordance with a predetermined training algorithm.
In another embodiment of the present invention, a second network model is provided which is operable to receive as inputs all of the input variables and provide predictive outputs associated with each of the output variables in the dataset. The second network model is trained on all of the input variables and output variables in the dataset to provide the learned representation of the output variables. The effect of each of the input variables to the network can then be determined on the learned representation of the output variables, with the network allowing the combined effects of all other input variables to be taken into account for determining the effect of any of the input variables on any one of the predicted output variables.
In yet another aspect of the present invention, a single neural network is utilized that has an input layer having a defined number of inputs for receiving input variables, an output layer for outputting a predetermined number of outputs and a hidden layer for mapping the input layer to the output layer. The neural network is configurable such that, in one mode, it can receive on the input layer all of the input variables and be trained on the entire dataset. In a second mode it can be configured to receive only the select ones of the input variables on the input layer and be trained only on the select portion of the dataset associated with the select ones of the input variables.
REFERENCES:
patent: 4893255 (1990-01-01), Tomlinson et al.
patent: 4912753 (1990-03-01), Evans, Jr.
patent: 4928484 (1990-05-01), Peczkowski
patent: 4979126 (1990-12-01), Pao et al.
patent: 5052043 (1991-09-01), Gaborski
patent: 5111531 (1992-05-01), Grayson
patent: 5113483 (1992-05-01), Keeler
patent: 5119468 (1992-06-01), Owens
patent: 5142612 (1992-08-01), Skeirik
patent: 5175678 (1992-12-01), Frerichs et al.
patent: 5283855 (1994-02-01), Motomura et al.
patent: 5825646 (1998-10-01), Keeler et al.
patent: 0 463 934 A1 (1992-02-01), None
patent: WO 90/10270 (1990-09-01), None
Caudill, M., “Using Neural Nets: Diagnostic Expert Nets Part 5”, AI Expert; Sep. 1990.
Hergert, F. et al., “A Comparison of Weight Elimation Methods for Reducing Complexity in Neural Networks”, IEEE 1992.
D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning Internal Representation by Error Propagation”, in Parallel Distribution Processing by D.E. Rumelhart and J.L. McClelland,
Hartman Eric Jon
Keeler James David
Liano Kadir
Garland Steven R.
Grant William
Howison, Chauza, Handley & Arnott, LLP
Pavilion Technologies, Inc.
LandOfFree
Method and apparatus for determining the sensitivity of... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for determining the sensitivity of..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for determining the sensitivity of... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2516012