Electrical computers and digital processing systems: multicomput – Computer-to-computer data routing – Least weight routing
Reexamination Certificate
1998-08-28
2001-04-24
Lee, Thomas (Department: 2783)
Electrical computers and digital processing systems: multicomput
Computer-to-computer data routing
Least weight routing
C709S223000, C709S235000
Reexamination Certificate
active
06223206
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Technical Field
This invention relates to the field of file servers, and in particular to a method and system for improving load balancing in a file server providing sequential file retrieval. More particularly, this invention relates to reducing disk bottlenecks due to fluctuating user demands generated for the continuous delivery in real time of large files including multimedia files, such as video movies.
2. Background Information
The concept of using file servers to allow users access to a large variety of data objects, including movies, has grown significantly over the last several years, with indications that such use will continue through the next decade. Fueling user demand is the potential ease of access to thousands upon thousands of different files, such as movie files.
In file servers, including large multimedia file servers, access to files, including video movie files, on storage devices, such as disks, may be limited by the number of access or retrieval streams capable of accessing the files on the storage devices. When a large number of users demand access to files on a disk, their access may require the use of all available retrieval streams to the disk, and a condition known as disk bottlenecking will occur. Essentially, with a bottlenecked disk, no bandwidth is available to service any further user requests from that disk. Disk bottlenecking, or as it is sometimes referred, load imbalancing, typically occurs when user demand for access to files on a disk is greater than the number of retrieval streams capable of accessing the files. When disk bottlenecking occurs a particular number of users desiring access to the files on the disk are incapable of accessing the desired files due to the unavailability of retrieval streams.
Avoiding such disk bottlenecks due to the imbalance in resource utilization in a multimedia server is critical because of the stringent response time requirements of sequentially based multimedia file server systems.
It is therefore desirable to reduce disk bottlenecks and load imbalances on large multimedia file servers so that greater user demand can be satisfied.
Disk bottlenecking or load imbalancing can be prevented by spreading each file over all the disks in the system. However, spreading each file over all the disks in the system will result in a time-consuming reconfiguration process when a new disk is added to or dropped (due to failure) from the system.
An alternative way to avoid disk bottlenecking is a static approach where all or some of the files in the system are replicated, in their entirety, onto different disks in the system. However, replicating some of the files requires advanced notice of the number of users desiring access to those files. If advanced notice of the number of users desiring access to certain files is known, then those “hot”, i.e. highly accessed, files may be replicated prior to retrieval by the users, thereby avoiding a disk bottleneck.
The problem, however, is that the number of users desiring access to certain files at a particular time is not always known and therefore obtaining advance notice of the same is not always possible. Therefore, it is very difficult to respond to user demand fluctuations.
Also, since disk replication space may be limited there may not be sufficient storage area for the replication of complete files on other disks. Furthermore, such space considerations are even more critical when one desires the replication of all files onto all disks. Finally, since the static replication of all the files over all the disks in the system requires considerable disk space, considerable time and cost also will be incurred.
These problems have prevented an adequate system for avoiding load imbalances in file servers, particularly multimedia file servers, which are used by end users to access movie files.
It is therefore desirable to provide an improved load balancing system.
It is also desirable to provide a method and system for improving load balancing in a file server capable of operating without advanced knowledge of the number of users desiring access to a file at a particular time.
It is also desirable to provide a method and system for improving load balancing in a file server where user demand fluctuates.
It is also desirable to provide a method and system for improving load balancing in a file server which is capable of detecting an overload condition while a file is being accessed by one or more retrieval streams.
It is also desirable to provide a method and system for improving load balancing in a file server which is capable of dynamically avoiding load imbalances as users attempt to access a particular file.
It is further desirable to provide a method and system for improving load balancing in a file server which is capable of replicating portions of accessed files onto other storage devices.
It is further desirable to provide a method and system for improving load balancing in a file server which is capable of permitting files to be read from a plurality of disks.
It is also desirable to provide a method and system for improving load balancing for the continuous delivery in real time of large multimedia files such as movies.
SUMMARY OF THE INVENTION
The aforementioned objects and advantages may be achieved through implementation of the method and system for improving load balancing in a file server in accordance with the principles of the present invention.
The invention involves a method for improving load balancing in a file server. The method includes determining the existence of an overload condition on a storage device having a plurality of retrieval streams for accessing at least one file thereon; selecting a first retrieval stream reading a file; replicating a portion of the file being read by the first retrieval stream onto a second storage device; and reading the replicated portion of the file on the second storage device with a retrieval stream capable of accessing the replicated portion of said file.
Determining the existence of an overload condition on a storage device may be accomplished by detecting a threshold level on the storage device. After an overload condition is detected, the first retrieval stream is selectively chosen from the plurality of retrieval streams reading the file. The second storage device may then be chosen from a plurality of storage devices, with the chosen storage device having the largest amount of storage space.
Replicating the portion of the file being read by the first retrieval stream onto the second storage device includes creating a switching point entry on a file information table so that the replicated portion of the file can be indexed for ease of switching between storage devices. Replicating the portion of the file onto the second storage device may comprise reading the file using the first retrieval stream from the first storage device, and then writing the file using the retrieval stream which is capable of accessing the replicated portions of the file onto the second storage device.
Each file may comprise a large number of blocks. Each file may also comprise a small number of segments, with each individual segment comprising a large number of blocks. Segments are preferably sized equally. Replication may involve replicating a plurality of blocks or replicating at least one segment.
Each file includes at least one primary file. Each primary file contains a plurality of primary segments, and primary segments are permanently stored segments. Permanent segments may not be deleted or overwritten. The replicated portions of a file comprise one or more secondary segments, and these secondary segments are temporarily stored segments. Temporary segments may be overwritten or deleted. Depending upon the load history of the storage devices involved, primary segments may be converted to secondary segments, and secondary segments may be converted to primary segments.
Replicating block portions of the file being read by the first retrieval stream onto the second storage device may be accomplished by sequen
Dan Asit
Kienzle Martin Gerhard
Sitaram Dinkar
Yu Philip Shi-Lung
Heslin & Rothenberg, P.C.
International Business Machines - Corporation
Lee Thomas
Park Ilwoo
LandOfFree
Method and system for load balancing by replicating a... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and system for load balancing by replicating a..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and system for load balancing by replicating a... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2524198