Electrical computers and digital processing systems: memory – Storage accessing and control – Specific memory composition
Reexamination Certificate
2002-07-01
2004-07-06
Bataille, Pierre (Department: 2186)
Electrical computers and digital processing systems: memory
Storage accessing and control
Specific memory composition
C707S793000, C711S170000, C711S202000, C714S006130
Reexamination Certificate
active
06760808
ABSTRACT:
FIELD OF THE INVENTION
The present invention is related to computer systems for capture, authoring and playback of multimedia programs and to distributed computing systems.
BACKGROUND OF THE INVENTION
There are several computer system architectures that support distributed use of data over computer networks. These computer system architectures are used in applications such as corporate intranets, distributed database applications and video-on-demand services.
Video-on-demand services, for example, typically are designed with an assumption that a user will request an entire movie, and that the selected movie has a substantial length. The video-on-demand server therefore is designed to support read-only access by several subscribers to the same movie, possibly at different times. Such servers generally divide data into several segments and distribute the segments sequentially over several computers or computer disks.
This technique commonly is called striping, and is described, for example, in U.S. Pat. Nos. 5,473,362, 5,583,868 and 5,610,841. One problem with striping data for movies over several disks is that failure of one disk or server can result in the loss of all movies, because every movie has at least one segment written on every disk.
A common technique for providing reliability in data storage is called mirroring. A hybrid system using mirroring and sequential striping is shown in U.S. Pat. No. 5,559,764 (Chen et al.). Mirroring involves maintaining two copies of each storage unit, i.e., having a primary storage and secondary backup storage for all data. Both copies also may be used for load distribution. Using this technique however, a failure of the primary storage causes its entire load to be placed on the secondary backup storage.
Another problem with sequentially striping data over several disks is the increased likelihood of what is called a “convoy effect.” A convoy effect occurs because requests for data segments from a file tend to group together at a disk and then cycle from one disk to the next (a “convoy”). As a result, one disk may be particularly burdened with requests at the one time while other disks have a light load. In addition, any new requests to a disk must wait for the convoy to be processed, thus resulting in increased latency for new requests. In order to overcome the convoy effect, data may be striped in a random fashion, i.e., segments of a data file is stored in a random order among the disks rather than sequentially. Such a system is described in “Design and Performance Tradeoffs in Clustered Video Servers,” by R. Tewari, et. al., in
Proceedings of Multimedia '
96, pp. 144-150. Such a system still may, experience random, extreme loads on one disk, however, due to the generally random nature of data accesses.
None of these systems is individually capable of transferring multiple, independent, high bandwidth streams of data, particularly isochronous media data such as video and associated audio data, between multiple storage units and multiple applications in a scalable and reliable manner. Such data transfer requirements are particularly difficult in systems supporting capture, authoring and playback of multimedia data. In an authoring system in particular, data typically is accessed in small fragments, called clips, of larger data files. These clips tend to be accessed in an arbitrary or random order with respect to how the data is stored, making efficient data transfer difficult to achieve.
SUMMARY OF THE INVENTION
Data is randomly distributed on multiple storage units connected with multiple applications using a computer network. The data is divided into segments. Each segment is copied, and each copy is stored on a different one of the storage units. The selection of each storage unit on which a copy of a segment is stored is random or pseudorandom and may be independent of the storage units on which other segments of the data are stored. Each segment is stored on at least two of the storage units.
This random distribution of multiple copies of segments of data improves both scalability and reliability. For example, when an application requests a selected segment of data, the request may be processed by the storage unit with the shortest queue of requests so that random fluctuations in the load applied by multiple applications on multiple storage units are balanced statistically and more equally over all of the storage units. Since the data is processed in terms of its segments, data fragments or clips also are processed as efficiently as all of the data. The applications may request data transfer from a storage unit only when that transfer would be efficient and may request storage units to preprocess read requests. In certain cases, bandwidth utilization on a computer network is optimized by scheduling data transfers among the clients and storage units. In addition, if one of the storage units fails, its load is distributed randomly and nearly uniformly over the remaining storage units. Procedures for recovering from failure of a storage unit also may be provided.
The storage units and applications also may operate independently and without central control. For example, each client may use only local information to schedule communication with a storage unit. Storage units and applications therefore may be added to or removed from the system. As a result, the system is expandable during operation.
This combination of techniques results in a system which can transfer multiple, independent high-bandwidth streams of data between multiple storage units and multiple applications in a scalable and reliable manner. Accordingly, in one aspect, a distributed data storage system includes a plurality of storage units for storing data, wherein copies of segments of data stored on the storage units are randomly distributed among the plurality of storage units. Each copy of each segment may be stored on a different one of the storage units. Each copy of each segment may be assigned to one of the plurality of storage units according to a probability distribution defined as a function of relative specifications of the storage units. The distributed data storage system may include a computer-readable medium having computer-readable logic stored thereon and defining a segment table accessible by a computer using an indication of a segment of data to retrieve indications of the storage units from the plurality of storage units on which the copies of the segment are stored. The plurality of storage units may include first, second and third storage units connected to a computer network.
In another aspect, a file system for a computer enables the computer to access remote independent storage units over a computer network in response to a request, from an application executed on the computer, to read data stored on the storage units. Copies of segments of the data are randomly distributed among the plurality of storage units. The file system is responsive to the request to read data, to select, for each segment of the selected data, one of the storage units on which the segment is stored. Each segment of the requested data is read from the selected storage unit for the segment. The data is provided to the application when the data is received from the selected storage units. In this file system, the storage unit may be selected such that a load of requests on the plurality of storage units is substantially balanced. The storage unit for the segment may be selected according to an estimate of which storage unit for the segment has a shortest estimated time for servicing the request.
More particularly, the file system may request data from one of the storage units, indicating an estimated time. If the first storage unit rejects the request, the file system may request data from another of the storage units, indicating another estimated time. The file system requests the data from the first storage unit when the second storage unit rejects the request. Each storage unit rejects a request for data when the request cannot be serviced by the storage unit wi
Fasciano Peter J.
Jacobs Herbert R.
Peters Eric C.
Rabinowitz Stanley
Avid Technology Inc.
Bataille Pierre
Gordon Peter J.
LandOfFree
Computer system and process for transferring multiple high... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Computer system and process for transferring multiple high..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Computer system and process for transferring multiple high... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3222732