Error detection/correction and fault detection/recovery – Data processing system error or fault handling – Reliability and availability
Reexamination Certificate
1997-10-17
2001-01-16
Kulik, Paul V. (Department: 2777)
Error detection/correction and fault detection/recovery
Data processing system error or fault handling
Reliability and availability
C714S018000
Reexamination Certificate
active
06175933
ABSTRACT:
The present invention relates to file transfers which read to or write from an input/output (I/O) device in a data processing system, and more particularly relates to recovery of file transfers in the event the transfer is stopped before the transfer is complete.
BACKGROUND OF THE INVENTION
A file transfer program is one in which a file is transferred between an application running on a computer and an I/O device in a read or write operation. The file being transferred comprises records grouped into blocks, wherein the blocking of the records is not important to the logical content of the file. The records may be blocked differently depending on where the file is stored. In a source file, the records may be blocked in one way, they may be blocked in another way while being transferred from the source to a destination, and they may be blocked in still another way when they are stored in the destination file.
Five specifications for file transfer programs which are difficult to support at the same time are:
1. A truly generic file transfer program allows users to specify their own I/O routines to read and write files. In this way, the file transfer program can read from and write to any I/O device for which the user has written an I/O routine.
Therefore, there is a requirement for an interface to a generic I/O routine that can handle many types of I/O.
2. When large amounts of data are being transferred, the file transfer program should allow for recovery in case the transfer is stopped for some reason. For example, a transfer might be stopped because a communication mechanism like TCP/IP has crashed, or because a machine must be recycled. When this occurs, it is often desirable to resume the transfer at a later time from the point at which it was stopped.
3. The file transfer program should support mechanisms for monitoring the transfer as it proceeds such as indicators of how far along the transfer is and at what rate it is transferring the file.
4. The file transfer program should support mechanisms for changing the contents of files as they are transferred such as stride, beginning and ending record number, and user supplied conversion routines.
5. The file transfer program should support transfers for file types which do not allow for recovery, such as named pipes useable with the IBM AIX operating system and batch pipes useable with the IBM MVS operating system. Files of these types do not allow a program to index into the middle of a file to start reading or writing from that point.
These five specifications work together to create many difficulties. For instance, at recovery time, parameters, referred to herein as status, used for monitoring the transfer must be set to the correct values so monitoring of status can be resumed. Also, the file changing algorithms (hereinafter referred to as the conversion routine) need to be reset to the correct value from the point at which the transfer must be resumed. Also, the values that the I/O routine must return for recovery may not correspond to the most recent values of the monitoring and file changing information. The I/O routine may need to recover to a point in the transfer well before the one to which the transfer had proceeded when it was stopped. Consider, for example, the file of
FIG. 1
which contains blocks
10
,
11
, and
12
. Block
10
includes records
1
-
6
, block
11
includes records
7
-
12
, and block
12
includes records
13
-
17
. A file transfer program will pass the records of this file to an I/O routine, one at a time, for writing to some storage medium. Each block will be placed into memory before it is written to the storage medium. When record
1
is passed to the I/O routine, the routine will place the record in a block of memory, but will not yet write the record to the storage medium. Thus, at this point the I/O routine cannot return recovery information for record
1
to the file transfer program because record
1
has not yet been written to the storage medium. Likewise, records
2
through
5
are passed by the file transfer program, and are also stored in the block of memory, but are still not moved to the storage medium. Thus, for records
1
through
5
, no recovery information can be returned to the file transfer program. Only when record
6
has been passed to the I/O routine can the routine store records
1
through
6
in the storage medium. It is then that recovery information can be sent to the file transfer program. The recovery information that is available at that time is the location on the storage medium at the beginning of the block, which is the location of the beginning of record
1
on the storage medium. An example of a file access method that provides this kind of information is the BSAM macros on the MVS operating system by IBM. The BSAM NOTE macro returns the position of the beginning of the block most recently written to disk or tape.
In a similar way, records
7
through
12
are written to the storage medium when record
12
is passed to the I/O routine. The file access method will return the position on the storage medium of record
7
, which the I/O routine then passes back to the file transfer program.
Consider the situation where records
13
through
15
are passed to the I/O routine, and are blocked in memory. If the file transfer stopped at this point for any reason, recovery must start with record
7
rather than record
16
because records
13
through
15
were never written to the storage medium, and thus cannot be recovered. Records
8
through
12
cannot be recovered because present access methods do not allow recovery to the middle of a block. In addition, though the file transfer's status information and file changing algorithms have progressed to record
15
, they must be reset to record
7
so that the transfer can be resumed from record
7
.
Some files do not allow a program to index into the middle of a file to start reading or writing. Recovery methods which involve indexing into the middle of both the source file and the destination file thus are not suitable for these types of files.
SUMMARY OF THE INVENTION
The present invention provides a general mechanism for specifying and keeping track of recovery information when writing to I/O devices that may not be able to immediately return information for a record.
When a record is passed to the I/O routine, the file transfer program saves all state information needed to recover that record. The state information consists of status information for monitoring, information needed when recovering file changing algorithms like stride and translation, and whatever else might be important about the current state of the transfer. An identifier is associated with the state information and passed to the I/O routine. The I/O routine must save this identifier until it has access method recovery information for the record. At that point, it returns both the access method recovery information and the identifier back to the file transfer program. The file transfer program can then keep track of both the access method recovery information and the state information so that if recovery becomes necessary, it can reset itself back to the appropriate state using the state information and tell the I/O routine to reset itself back to the appropriate state using the access method recovery information. In addition, if the I/O routine needs to save additional state information besides the access method recovery information, it can return this state information with the access method recovery information to the file transfer program to be saved. Records that cannot be recovered are stored in memory until the transfer reaches a point at which they can be recovered. Thus, the file transfer program does not index into the middle of the source file thereby providing for recovery of file types that do not support such indexing.
It is a primary object of the present invention to provide a file transfer program which may recover records previously transferred in the event the file transfer program stops.
It is another object of t
Cutter Lawrence D.
Gonzalez Floyd A.
International Business Machines - Corporation
Kulik Paul V.
LandOfFree
Recovery of file transfers in a data processing system does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Recovery of file transfers in a data processing system, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Recovery of file transfers in a data processing system will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2438982