Data processing: artificial intelligence – Neural network – Learning task
Reexamination Certificate
1997-09-22
2001-07-10
Powell, Mark R. (Department: 2122)
Data processing: artificial intelligence
Neural network
Learning task
C706S023000, C706S048000
Reexamination Certificate
active
06260032
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a method for structuring a computing module for learning non-linear mapping which is generally a multivalued mapping. More specifically, the invention relates to a method for learning multivalued mapping by a functional approximation which enables learning to obtain an overview of the multifold structure of the mapping from a small amount of example data and also enables learning local changes.
2. Description of the Related Art
The inventor of the present invention proposed a method for multivalued function approximation (see Japanese Patent Application Laid Open (kokai) No. 7-93296, “Method for Learning Multivalued Function” and Transactions of the Institute of Electronics, Information and Communication Engineers of Japan A, vol. J78-A, No. 3, pp. 427-439 (1995) “Multivalued Regularization Network”).
However, in the above-mentioned method for multivalued function approximation, only locally defined base functions are used, although multivalued functions can be expressed. Namely, F
k
is expressed by a function of a linear sum of the local base functions centering on {right arrow over (t)}
kp
, as follows:
F
k
(
x
→
)
=
∑
p
=
1
M
r
kp
K
kp
(
x
→
,
t
→
kp
)
(
1
)
The above mentioned Japanese Patent Application Laid Open No. 7-93296 describes an example wherein the center {right arrow over (t)}
kp
coincides with the input portion {right arrow over (x)}
(i)
of the teaching data.
However, in the above-mentioned method for multivalued function approximation, since base functions are defined only in the vicinity of the teaching data or the center {right arrow over (t)}
kp
, functions are not defined where the teaching data do not exist. Another problem is that when the numbers of input and output space dimensions (m+n) become large the required amount of teaching data remarkably increases.
SUMMARY OF THE INVENTION
An object of the present invention is to solve the above-mentioned problems and to provide a method for learning multivalued mapping.
The method for learning multivalued mapping according to the present invention provides a method for approximation of a manifold in (n+m)-dimensional space expressing a mapping computation module having n inputs and m outputs.
According to a first aspect of the present invention, there is provided a method for learning multivalued mapping for providing a method for approximation of manifold in (n+m)-dimensional space, by learning a smooth function from n-dimensional input space to m-dimensional output space which optimally approximates m-dimensional vector value data forming a given plurality of layers in n-dimensional space, the method comprising steps of:
(a) mathematically expressing a multivalued function directly in Kronecker's tensor product form;
(b) developing and replacing the functions so as to obtain linear equations with respect to unknown functions;
(c) defining the sum of a linear combination of local base functions and a linear combination of polynomial bases with respect to the replaced unknown function; and
(d) learning from example data, an manifold which is defined by the linearized function in the input-output space, through use of a procedure for optimizing the error and the smoothness constraint.
According to a second aspect of the present invention, there is provided a method for learning multivalued mapping for providing a method for approximation of manifold in (n+m)-dimensional space, by obtaining a smooth function from n-dimensional input space to m-dimensional output space which optimally approximates m-dimensional vector value data forming a given plurality of layers in n-dimensional space, the method comprising steps of:
(a) expressing a manifold by synthesizing h m-dimensional vector value functions in n-dimensional space according to the following equation:
{
{right arrow over (y)}−f
1
(
{right arrow over (x)}
)}
{
{right arrow over (y)}−f
2
(
{right arrow over (x)}
)}
. . .
{
{right arrow over (y)}−f
h
(
{right arrow over (x)}
)}=0 (2)
wherein “
” denotes Kronecker's tensor product,
{right arrow over (x)}
=(
x
1
,x
2
, . . . x
n
)
T
,
{right arrow over (y)}
=(
y
1
,y
2
, . . . y
m
)
T
,
and “T” denotes transposition of a vector;
(b) developing the above equation and converting it into a linear equation with respect to unknown functions;
(c) expressing each unknown function as:
F
k
(
x
→
)
=
∑
p
=
1
M
r
kp
K
kp
(
x
→
,
t
→
kp
)
+
∑
j
=
1
d
k
s
kj
φ
kj
(
x
→
)
;
(
3
)
(d) defining an error functional for calculating the error on the left side of Equation (2) for calculating the unknown function F
k
from the example data;
(e) defining a regularizing functional as the square of the absolute value of the result of the operation in which the operator defining the smoothness constraint of the unknown function is applied to each unknown function, as required,
(f) minimizing the error functional and the regularizing functional, and deriving a procedure for obtaining the unknown functions F
k
; and
(g) obtaining a conversion function for calculating f
j
from the unknown functions F
k
by formula manipulation method or by numerical approximation algorithm.
In the method for learning multivalued mapping according to the second aspect, m may be 1 or 2.
In the method for learning multivalued mapping according to the second aspect, the equality {right arrow over (t)}
0p
={right arrow over (t)}
1p
={right arrow over (t)}
2p
= . . . ={right arrow over (t)}
hp
may be satisfied.
In the method for learning multivalued mapping according to the second aspect, N pairs of example data may be [({right arrow over (x)}
(i)
, {right arrow over (y)}
(i)
)|i=1, 2, . . . , N], and M=N, {right arrow over (t)}
kp
={right arrow over (x)}
(p)
(wherein p=1, 2, . . . , N).
In the method for learning multivalued mapping according to the second aspect, the equality K
k1
=K
k2
= . . . =K
kM
(wherein k=0, 1, 2, . . . , h) may be satisfied.
REFERENCES:
patent: 5083285 (1992-01-01), Shima et al.
patent: 5268834 (1993-12-01), Sanner et al.
patent: 5276771 (1994-01-01), Manukian et al.
patent: 5416899 (1995-05-01), Poggio et al.
patent: 5446828 (1995-08-01), Woodall
patent: 5568590 (1996-10-01), Tolson
patent: 5613041 (1997-03-01), Keeler et al.
patent: 5729660 (1998-03-01), Chiabrera et al.
patent: 5774631 (1998-06-01), Chiabrera et al.
patent: 5987444 (1999-11-01), Lo
patent: 6173218 (2001-01-01), Vian
Vijaykumar et al, “Fast and efficient incremental learning for high dimensional movement systems” ICRA-IEEE, pp 1894-1899, Apr. 2000.*
Lee et al< “Indentification and control of dynamic systems using recurrent fuzzy neural networks”, Trans. of Fuzzy Systems IEEE, pp 349-366, vol. 8, No. 4, Aug. 2000.*
Piovoso et al, “Neural network process control”, ACM, pp 84-94, May 1991.*
Hu et al, “Approximation the longest approximate common subsequence problem”, ACM pp 166-172, Sep. 1998.*
Koiran, “Efficient learning of contineous neural network”, COLT ACM, pp 348-355, Apr. 2000.*
M. Shizawa, “Multi-Valued Regularization Network: A Theory of Multi-Layer Networks for Learning Many-to-H Mappings,” Journal of the Institute of Electronics, Information, and Communication Engineers [English Translation], vol. J78-A, No. 3, pp. 427-439, Mar. 1995.*
M. Shizawa, “Regularization Networks for Approximating Multi-Valued Functions: Learning Ambiguous Input-Output Mappings from Examples,” 1994 IEEE Int. Conference on Neural Networks, vol. 1, pp. 137-142, Jul. 1994.*
M. Shizawa, “Multi-Valued Regularization Network—A Theory of Multi-layer Networks for Learning Many-to-h Mappings” (abstract), Transactions of the Institute of Electronics, Information, and Communicati
Japan Science and Technology Corporation
Khatri Anil
Lorusso & Loud
Powell Mark R.
LandOfFree
Method for learning multivalued mapping does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method for learning multivalued mapping, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method for learning multivalued mapping will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2498911