Data processing: database and file management or data structures – Database design – Data structure types
Reexamination Certificate
1999-07-15
2001-12-11
Alam, Hosain T. (Department: 2172)
Data processing: database and file management or data structures
Database design
Data structure types
C203S014000, C203S100000, C203S100000
Reexamination Certificate
active
06330572
ABSTRACT:
TECHNICAL FIELD
The present invention relates to data storage and, more particularly, to systems and methods for hierarchical storage management.
BACKGROUND
Hierarchical storage management (HSM) systems allow the managed storage of data files among a variety of media such as magnetic hard disks, magneto-optic disks, and magnetic tape. The various media differ in access time, capacity, and cost. Thus, HSM systems typically are configured such that files that are accessed more frequently or created more recently are stored on “short-term” media having the shortest access time. Short-term media often includes a group of magnetic disks, which may be arranged as a redundant array of independent disks (RAID). Files that are accessed less frequently, created less recently, or have larger sizes are stored on “long-term” media having longer access times and larger storage capacities. Long-term media in an HSM system may include rewritable optical disks and magnetic tape media, which can be arranged in a jukebox of magneto-optical disks or a tape library, respectively.
Existing HSM systems typically allocate individual files across the hierarchy of storage media based on frequency of use, creation date, or file size, as discussed above. Accordingly, HSM systems generally seek to avoid excessive access delays in retrieving information that is likely to be accessed most often or most frequently by users. As new files are generated, the system stores the files on the short-term media using a “best-fit” approach. In this manner, the system distributes files across the short-term media in order to minimize wasted storage space. Thus, each file may be stored on a different medium in order to most efficiently manage storage space.
A central database maintains the storage location of each file within the HSM system. If users do not request a particular file for an extended period of time, the system automatically migrates the corresponding file to the longer-term storage media and updates the file location database. Again, the system distributes the relocated file across the long-term storage media in a manner calculated to minimize wasted storage space. For image files, an HSM system may store a number of copies at different display resolutions and on different media to facilitate identification and retrieval by a user. When a user requests a particular file, the system accesses the database to determine the current location of the file. If the desired files reside on longer-term storage media, the system automatically retrieves the files and moves them to the short-term media. If some of the media is not currently loaded into the longer-term storage device, the system generates a request for personnel to physically locate the media and load it into the storage device.
SUMMARY
The present invention is directed to a system and method for managing the storage of files within an HSM system. The system and method are especially useful in managing the storage of larger files that include graphic imagery. The system and method may incorporate an architecture and methodology that facilitate the storage and retrieval of image files as part of an overall image processing workflow. In particular, the system and method may find ready application in a workflow that involves the processing of groups of images associated with particular customers, projects, or transactions, and may act as a storage server for a client application that implements the workflow.
The system and method may be useful, for example, in handling the storage of images uploaded from scanned photographic film, or digital images submitted to a photo-processing shop by amateur or professional photographers. In this case, the client application can be a photo-processing application that could provide for various media formats, sizes, and quantities of image reproductions for a consumer. As another example, the system and method may be useful in handling the storage of medical diagnostic images associated with a particular medical patient or study. In this case, the client application could be a picture archival communication system (PACS) that manages archival of imagery for viewing by physicians. Further, the system and method may be useful in handling the storage of images associated with particular printing jobs, e.g., for publishers, advertising customers, and the like. In this case, the client application could be a digital prepress workflow application.
The system and method may incorporate a number of architectural and functional features capable of improving overall economy of storage while maintaining workflow efficiency for the user, as described below. Such features may be used together or independently of one another according to the needs of the particular storage and workflow applications.
As an example, a fileset feature can be provided that permits grouping of images associated with a particular customer, project, or transaction. The fileset feature allows a user to group files into a logical collection, and perform operations on the files as a group. Moreover, filesets can be configured to have their member files reside together on common media, so that operations on the fileset such as archival, migration, retrieval, and deletion can be performed significantly faster than operating on the individual files, which might otherwise be distributed across multiple media.
In this manner, the system and method are capable of intelligently grouping images for storage together on common storage media to alleviate excessive access times in retrieving the images. Thus, the fileset feature can be used to avoid the scattering of associated images across diverse storage media having different access times, and resulting delays in incorporating such images into the workflow of the client application. In addition, a fileset can be configured such that activity on an individual file member triggers the same activity on the other file members of the fileset.
Also, a media independence feature can be incorporated whereby data can be stored on particular volumes without knowledge of the media type of the volume. In this case, the system and method provide an application programming interface that is substantially identical for all media types, for both direct access and sequential access, and for removable and non-removable media.
Along with media independence, the system and method may further include a feature that allows the storage volumes to support self-describing media. Normally, an inventory of the contents of a volume are maintained as entries in a database. This self-describing media feature provides the capability to reconstruct the database entries for the files stored on a volume when a volume is physically moved to another system that is not on the same network. A volume may be physically moved, for example, when the database has been corrupted, when identifying unlabeled or mislabeled media from shelf storage, or when a volume is moved to a remote server.
The self-describing media feature can be implemented by storing volume metadata on each physical volume and by storing file metadata for each data file on the same physical volume on which the data file is stored. Thus, each file may include two files: a blob file with the actual file contents and a metadata file with identifying information. The metadata provides sufficient information to rebuild an inventory of the contents of a volume without access to the original file location database.
The metadata can be useful in tracking volumes and files, verification of the identities of loaded volumes, and database recovery. The metadata can be specified by a client application and can be substantially transparent to the storage management server, providing information known to the client application such as the particular client applications and users associated with the volume or file.
For unique identification of volumes and files, the metadata may incorporate a global unique identifier (guid) that is unique on a single server, as well as across a multi-server system. Also, a
Alam Hosain T.
Bauer William D.
Imation Corp.
Truong Cam-Y
LandOfFree
Hierarchical data storage management does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Hierarchical data storage management, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Hierarchical data storage management will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2594132