Data processing: artificial intelligence – Neural network – Structure
Reexamination Certificate
2002-02-06
2004-09-14
Davis, George B. (Department: 2121)
Data processing: artificial intelligence
Neural network
Structure
C706S016000
Reexamination Certificate
active
06792413
ABSTRACT:
BACKGROUND OF THE INVENTION
The present invention relates generally to a data processing apparatus and method, a recording medium, and a program and, more particularly, to a data processing apparatus and method, a recording medium, and a program which can easily and infallibly store and recall complicated time-series data.
The applicant hereof disclosed in Japanese Patent Laid-open No. Hei 11-126198 a technology of generating time-series data by use of a neural network of recurrent type.
In the disclosed technology, as shown in
FIG. 1
, the apparatus is basically configured with a lower-layer network having recurrent neural networks (RNNs)
1
-
1
through
1
-n and a higher-layer network having recurrent neural networks RNNs
11
-
1
through
11
-n.
In the lower-layer network, the outputs of the RNNs
1
-
1
through
1
-n are supplied to a combining circuit
3
via respective gates
12
-
1
through
12
-n.
In the higher-layer network, the outputs of the RNNs
11
-
1
through
11
-n are supplied to a combining circuit
13
via respective gates
12
-
1
through
12
-n. In accordance with the a combined output from the combining circuit
13
of the higher-layer network, the on/off operations of gates
2
-
1
through
2
-n of the lower-layer network are controlled.
The RNNs
1
-
1
through
1
-n of the lower-layer network generate patterns P
1
through Pn respectively. On the basis of the output of the combining circuit
13
of the higher-layer network, predetermined one of the gates
2
-
1
through
2
-n of the lower-layer network is turned on/off, thereby causing the combining circuit
3
to selectively output one of the patterns P
1
through Pn generated by the predetermined one of the RNNs
1
-
1
through
1
-n.
Consequently, as shown in
FIG. 2
for example, patterns which change with time can be generated by generating pattern P
1
for a predetermined period and then pattern P
2
for another predetermined period and then pattern P
1
again for still another predetermined period, for example.
However, in the above-mentioned disclosed technology, the gates
2
-
1
through
2
-n executes a so-called winner-take-all operation, so that it is difficult to store and generate complicated patterns.
SUMMARY OF THE INVENTION
It is therefore an object of the present invention to provide a data processing apparatus and method, a recording medium, and a program which are capable of easily and infallibly store and generate patterns even though they are complicated.
In carrying out the invention and according to a first aspect thereof, there is provided a data processing apparatus including: processing means including a first recurrent neural network for applying a predetermined nonlinear function to data inputted from an input terminal, the processing means receiving first data constituted by time-series data and second data constituted by time-series data at the input terminal of the first recurrent neural network to execute the processing; generating means including a second recurrent neural network for generating the second data by applying a predetermined nonlinear function to data inputted from an input terminal; and computing means for executing computation on the second data and third data generated by error back propagation by the first recurrent neural network to generate fourth data.
Preferably, in the above-mentioned data processing apparatus, the generating means generates the second data which change with a longer period than that of the first data.
Preferably, in the above-mentioned data processing apparatus, the computing means executes computation by use of data generated by error back propagation by the first recurrent neural network at the time of learning.
Preferably, in the data processing apparatus, the computing means executes the computation by use of a sigmoid function.
Preferably, in the data processing apparatus, the computing means executes, at the time of learning, a computation including a first computation using data generated by error back propagation by the first recurrent neural network and a second computation for smoothing in an adjacent space-time.
Preferably, in the data processing apparatus, the computing means executes, at the time of future prediction, a computation including a first computation of the second data and a second computation for smoothing in an adjacent space-time.
Moreover, this computing means may execute, at the time of recalling the past, a computation including a first computation of the second data, a second computation using data generated by error back propagation by the first recurrent neural network, and a third computation for smoothing in an adjacent space-time.
In carrying out the invention and according to a second aspect thereof, there is provided a data processing method including: a processing step for performing processing by using a first recurrent neural network for applying a predetermined nonlinear function to data inputted from an input terminal, the processing step receiving, at the input terminal of the first recurrent neural network, first data constituted by time-series data and second data constituted by time-series data to execute the processing of the first and second data; a generating step for performing processing by using a second recurrent neural network for generating the second data by applying a predetermined nonlinear function to data inputted from an input terminal; and a computing step for executing computation on the second data and third data generated by error back propagation by the first recurrent neural network to generate fourth data.
In carrying out the invention and according to a third aspect thereof, there is provided a recording medium recording a computer-readable program, including: a processing step for performing processing by using a first recurrent neural network for applying a predetermined nonlinear function to data inputted from an input terminal, the processing step receiving, at the input terminal of the first recurrent neural network, first data constituted by time-series data and second data constituted by time-series data to execute the processing of the first and second data; a generating step for performing processing by using a second recurrent neural network for generating the second data by applying a predetermined nonlinear function to data inputted from an input terminal; and a computing step for executing computation on the second data and third data generated by error back propagation by the first recurrent neural network to generate fourth data.
In carrying out the invention and according to a fourth aspect thereof, there is provided a program for causing a computer to execute: a processing step for performing processing by using a first recurrent neural network for applying a predetermined nonlinear function to data inputted from an input terminal, the processing step receiving, at the input terminal of the first recurrent neural network, first data constituted by time-series data and second data constituted by time-series data to execute the processing of the first and second data; a generating step for performing processing by using a second recurrent neural network for generating the second data by applying a predetermined nonlinear function to data inputted from an input terminal; and a computing step for executing computation on the second data and third data generated by error back propagation by the first recurrent neural network to generate fourth data.
In the data processing apparatus and method and program associated with the present invention, the second data generated by the second recurrent neural network is supplied to the input terminal of the first recurrent neural network to be processed together with the first data.
REFERENCES:
patent: 5182794 (1993-01-01), Gasperi et al.
patent: 5485545 (1996-01-01), Kojima et al.
patent: 5649065 (1997-07-01), Lo et al.
patent: 5732382 (1998-03-01), Puskorius et al.
patent: 5828812 (1998-10-01), Khan et al.
patent: 6041299 (2000-03-01), Schuster et al.
patent: 6199057 (2001-03-01), Tawel
patent: 6434084
Davis George B.
Frommer William S.
Frommer & Lawrence & Haug LLP
Sony Corporation
LandOfFree
Data processing apparatus and method, recording medium, and... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Data processing apparatus and method, recording medium, and..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Data processing apparatus and method, recording medium, and... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3255317