Data processing: artificial intelligence – Neural network – Neural simulation environment
Reexamination Certificate
1998-10-19
2001-08-07
Powell, Mark R. (Department: 2122)
Data processing: artificial intelligence
Neural network
Neural simulation environment
C706S045000
Reexamination Certificate
active
06272480
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention is directed to a method and to an arrangement for modelling a system with non-linear stochastic behavior, particularly a biological system such as, for example, the insulin-glucose metabolism, for which, as well as for other systems, only a small amount of training data is available for training the neural network.
2. Description of the Prior Art
Since measurements of influencing variables for the determination of the state of a technical or physiological system are very involved and complicated to implement, they are often only undertaken at irregular time intervals. For example, a diabetic person only determines his or her blood sugar content 4-5 times a day. If one attempts to produce models of such systems, an added complication is that these behave highly non-linearly and stochastically, so that computerized neural networks seem suitable for their modelling. Such computerized networks are usually utilized in “free-running” mode or in the “teacher-forcing mode” in which current measurements of the time series that is made available to the network replace iterated values. Both approaches are problematical in systems that behave highly stochastically and wherein only a few measured values in the time series are available for the individual influencing variables. It is known from “Lewis, F. L. (1986) Optimal Estimation, John Wiley, N.Y.” to approach such problems wih the assistance of stochastic models in which, for example, non-linear condition-space models are employed. However, there is still the problem of predicting and training lacking measured values whose analytical solution leads to such complicated integrals that they are unmanageable. Alternatively thereto, condition-dependent linearizations can be implemented for the prediction and the training, the most popular thereof being the “Extended Kalman Filter”. Other possible solutions for such problems are not known in the prior art.
SUMMARY OF THE INVENTION
An object of the present invention is to provide a method and an arrangement employing a computerized neural network in order to obtain a valid model of systems that behave non-linearly and stochastically, wherein few measured values of the influencing variables for such systems are available for the training of computerized neural network.
The above object is achieved in accordance with the principles of the present invention in a method and an apparatus for the neural modeling of a dynamic system with non-linear stochastic behavior, wherein the system behavior of a dynamic system is modeled in the form of a time series composed of at least one influencing variable of the system in order to make a prediction of that variable, wherein the influencing variable is formed as an additive combination of a deterministic output quantity of a computerized recurrent neural network and a linearly modeled system error, and wherein the computerized recurrent neural network is adapted at a first point in time with an error model adaption error formed as a difference between the influencing variable of the system measured at the first point in time and the linearly modeled system error.
An advantage of the inventive method and of the inventive arrangement is that simple iteration rules with which the linear error model can be improved are obtained for single-step or multi-step prediction by the combination of a linear error model with a recurrent neural network. A further advantage is that the computerized neural network can be trained with the assistance of real time recurrent learning for the maximum likelihood leaming, and that the linear error model can be trained with the assistance of an adaption rule that makes use of the forward or backward Kalman filter equations.
In an embodiment of the method, the specified system equations are advantageously employed since this involves an optimally low calculating outlay in the training and in the simulation of the model.
In an embodiment of the method, the dynamic system of the glucose-insulin metabolism of a diabetes patient can be advantageously modelled, whereby the glucose level of the patient is preferably modelled by the computerized neural network as the influencing variable and the error model is trained, since few values are available for this measured value as the influencing variable and the overall system behaves highly stochastically and non-linearly.
In a further embodiment of the method, time series of administered insulin doses, of the quantity of food, physical training and the current as well as the preceding estimated blood sugar value are made available for training the model, since these are the influencing variables that most often influence the blood sugar level.
An arrangement for the computerized neural modelling of a dynamic system with non-linear stochastic behavior includes a computerized recurrent network as well as means for error modelling of the system error of the dynamic system, the computerized neural network being trained with the difference between the system error and a measured value that was measured at the respective point in time of the time series. In this way, it is assured that the neural network learns all non-linearities of the system.
In an embodiment of the arrangement, the computerized neural network is implemented as a multi-layer perceptron since such neural networks are especially well-suited for modelling incomplete time series.
An embodiment of the arrangement is advantageously utilized for predicting the blood sugar value of a diabetes patient in that the current blood sugar value is made available, since the best prediction results can currently be achieved with such an arrangement.
REFERENCES:
patent: 5706401 (1998-01-01), Tresp
patent: 5748848 (1998-05-01), Tresp
patent: 5877954 (1999-11-01), Klimasauskas
patent: 5987444 (1999-11-01), Lo
“Optimal Estimation With An Introduction To Stochastic Control Theory,” Lewis (1986), pp. 249-272.
“Stochastic Processes and Filtering Theory,” Jazwinski (1970), pp. 200-219.
“An Approach To Time Series Smoothing And Forecasting Using The EM Algorithm,” Shumway et al., Technical Report No. 27, Division of Statistics, UC Davis (1981).
“Neural Network Modeling of Physiological Processes,” Tresp et al., Learning Theory And Natural Learning Systems 2, Hanson et al., eds., MIT Press (1994), pp. 363-378.
Puskorius, G.V.; Feldkamp, L.A., Practical considerations for Kalman filter training of recurrent neural networks, Neural Networks, 1993., IEEE International Conference on, 1993, pp.: 1189-1195 vol. 3, Jan. 1993.*
Puskorius, G.V.; Feldkamp, L.A., Truncated backpropagation through time and Kalman filter training for neurocontrol, Neural Networks, 1994. IEEE World Congress on Computational Intelligence., 1994 IEEE International Conference on, vol.: 4, 1994, pp., Jan. 1994.*
Puskorius, G. V.; Feldkamp, L.A., Model reference adaptive control with recurrent networks trained by the dynamic DEKF algorithm, Neural Networks, 1992. IJCNN., International Joint Conference on, vol.: 2, 1992, pp.: 106-113 vol. 2, Jan. 1992.*
Puskorius, G. V.; Feldkamp, L.A., Decoupled extended Kalman filter training of feedforward layered networks, Neural Networks, 1991., IJCNN-91-Seattle International Joint Conference on, vol.: i, 1991, pp.: 771-777 vol. 1, Jan. 1991.*
Puskorius, G.V.; Feldkamp, L.A., Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks, Neural Networks, IEEE Transactions on, vol.: 5 2 , Mar. 1994, pp.: 279-297, Jan. 1994.*
Feldkamp, L.A.; Puskorius, G.V.; Davis, L.I., Jr.; Yuan, F., Strategies and issues in applications of neural networks, Neural Networks, 1992. IJCNN., International Joint Conference on, vol.: 4, 1992, pp.: 304-309 vol. 4, Jan. 1993.
Briegel Thomas
Tresp Volker
Powell Mark R.
Schiff & Hardin & Waite
Siemens Aktiengesellschaft
Starks W.
LandOfFree
Method and arrangement for the neural modelling of a dynamic... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and arrangement for the neural modelling of a dynamic..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and arrangement for the neural modelling of a dynamic... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2445007