Error detection/correction and fault detection/recovery – Data processing system error or fault handling – Reliability and availability
Reexamination Certificate
2000-10-04
2003-11-25
Beausoliel, Robert (Department: 2184)
Error detection/correction and fault detection/recovery
Data processing system error or fault handling
Reliability and availability
C714S005110, C714S006130, C714S006130, C714S006130, C711S161000, C711S162000, C707S793000
Reexamination Certificate
active
06654912
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
The invention relates to recovery of file system data in file servers having mirrored file system volumes.
2. Related Art
Network file servers and other file systems are subject to errors and other failures, including those arising from hardware failure, software error, or erroneous configuration. Because of the possibility of error, many file systems provide additional copies of data in the file system, such as by providing a mirrored file system volume. In a mirrored file system, a first volume provides a first copy of the file system, while a second volume provides a synchronous, second copy of the file system. Thus, if data on the first volume is corrupted or otherwise lost, data from the second volume can be used in its place transparently.
One problem in the known art is that the first volume and second volume of the file system can fail to remain in synchronization. Thus, each volume of the mirrored file system would include a set of files or other objects from a different timestamp (or checkpoint) in the file system history. As a result, the first volume and second volume will no longer serve as accurate mirrors for each other because one is out-of-date. An aspect of this problem is that, after system crashes, it is unknown which of the first volume and second volume is the most recent. Accordingly, it would be desirable to assure that the first volume and second volume of the file system remain synchronized after system crashes. If it is not possible for the first volume and second volume to remain synchronized, it is desirable to rapidly determine which is the most recent version and use efficiently, so as to cause resynchronization.
A first known method is to resynchronize the two mirror copies after system crashes by copying every block. While this method can generally achieve the result of assuring that the first copy and second copy of the file system are synchronized after system crashes, it has the severe drawback that it is very inefficient, as each file block of at least one of the mirror file systems must be copied to the other one of the mirror file systems. When the file system is particularly large, such as one that approaches or exceeds a terabyte in size, this drawback makes this known method untenable due to its incredible latency (and liability to other failures).
A second known method is to maintain a log of regions or file blocks in each mirrored volume that have been changed (sometimes known as “dirty” file blocks). When such a log is maintained, it is only necessary to copy those file blocks that are dirty, rather than an entire mirrored volume. While this method can generally achieve the result otherwise achieved by the first known method, is still subject to at least two drawbacks. First, this method is more complex, in that it requires careful maintenance so as to ensure that the log remains synchronous. Second, the log itself must generally be mirrored for reliability, which of course re introduces the entire problem of recovery of mirrored files after system crashes. Third, maintaining this additional log increases the latency of every operation. Moreover, such a technique can introduce additional errors in the event that the log is unreliable.
Accordingly, it would be desirable to provide a technique for recovery of file system data in file servers having mirrored file system volumes that is not subject to drawbacks of the known art.
SUMMARY OF THE INVENTION
The invention provides a method and system for recovery of file system data in file servers having mirrored file system volumes. In a preferred embodiment, the invention makes use of a consistency point model including a snapshot feature of a robust file system (the “WAFL File System”), such as disclosed in the Incorporated Disclosures, to rapidly determine which of two or more mirrored volumes is most up-to-date, and which blocks of the most recent mirrored volume have been changed from each one of the mirrored file systems. Among a plurality of two or more mirrored volumes, the invention rapidly determines which is the most up-to-date by examining a most recent consistency point number maintained by the WAFL File System at each mirrored volume. The invention rapidly and reliably determines what blocks are shared between that most up-to-date mirrored volume and each other mirrored volume, in response to a snapshot of the file system maintained at each mirrored volume and are stored in common pairwise between each mirrored volume and the most up-to-date mirrored volume. The invention copies only those blocks that have been changed between the common snapshot and the most up-to-date snapshot. This rapid and reliable comparison of blocks, followed by the efficient transfer of those blocks that have been changed, does not present drawbacks of the known art.
The invention provides an enabling technology for a wide variety of applications for file system recovery using redundant file systems, so as to obtain substantial advantages and capabilities that are novel and non-obvious in view of the known art. Examples described below primarily relate to mirrored file system volumes in a network file server, but the invention is broadly applicable to many different types of redundant file systems, such as those used in RAID subsystems and parallel storage systems.
REFERENCES:
patent: 4761785 (1988-08-01), Clark et al.
patent: 4875159 (1989-10-01), Cary et al.
patent: 4897781 (1990-01-01), Chang et al.
patent: 4937763 (1990-06-01), Mott
patent: 5067099 (1991-11-01), McCown et al.
patent: 5163148 (1992-11-01), Walls
patent: 5208813 (1993-05-01), Stallmo
patent: 5235601 (1993-08-01), Stallmo et al.
patent: 5276867 (1994-01-01), Kenley et al.
patent: 5305326 (1994-04-01), Solomon et al.
patent: 5313626 (1994-05-01), Jones et al.
patent: 5335235 (1994-08-01), Arnott
patent: 5355453 (1994-10-01), Row et al.
patent: 5357509 (1994-10-01), Ohizumi
patent: 5604862 (1997-02-01), Midgely et al.
patent: 5649152 (1997-07-01), Ohran et al.
patent: 5649196 (1997-07-01), Woodhill et al.
patent: 5666353 (1997-09-01), Klausmeier et al.
patent: 5819292 (1998-10-01), Hitz et al.
patent: 5819310 (1998-10-01), Vishlitzky et al.
patent: 5835953 (1998-11-01), Ohran
patent: 5873101 (1999-02-01), Klein
patent: 5907672 (1999-05-01), Matze et al.
patent: 5948110 (1999-09-01), Hitz et al.
patent: 5963962 (1999-10-01), Hitz et al.
patent: 5996086 (1999-11-01), Delaney et al.
patent: 5996106 (1999-11-01), Seyyedy
patent: 6101585 (2000-08-01), Brown et al.
patent: 6119244 (2000-09-01), Schoenthal et al.
patent: 6182198 (2001-01-01), Hubis et al.
patent: 6212531 (2001-04-01), Blea et al.
patent: 6341341 (2002-01-01), Grummon et al.
patent: 6397307 (2002-05-01), Ohran
patent: 6549921 (2003-04-01), Ofek
patent: 0 462 917 (1991-12-01), None
patent: 0 492 808 (1992-07-01), None
patent: 0537098 (1993-04-01), None
patent: 0 552 580 (1993-07-01), None
patent: 0 767 431 (1997-04-01), None
patent: WO 94/29796 (1994-12-01), None
patent: WO 99/66401 (1999-12-01), None
patent: WO 00/07104 (2000-02-01), None
patent: WO 01/31446 (2001-05-01), None
patent: WO 01/43368 (2001-06-01), None
Jim Gray et al., “Parity Striping of Disc Arrays: Low-Cost Reliable Storage with Acceptable Throughput”, Tandem Computers Inc., 19333 Vallco Parkway, Cupertino, CA, International Conference on Very Large Data Bases, 16th. Int. Conference, Aug. 13-16, 1990, Brisbane, AU (1990) Aug. 13-16, New York, US. XP000522459.
Dave Hitz, et al., “File System Design for an NFS File Server Appliance”, Technical report 3002, Rev. C 3/95, Presented Jan. 19, 1994, USENIX Winter 1994, San Francisco, California.
David Hitz, “An NFS File Server Appliance”, Technical Report TR01, Rev. A 8/93, Network Appliance Corporation, Santa Clara, California.
Steven R. Kleiman, “Using NUMA Interconnects for Highly Available Filers”, 1999 IEEE.
Jai Menon et al., “The Architecture of a Fault-Tolerant Cached RAID Controller”, IBM Almaden Research Center, San Jose, California, 1993 IEEE, XP000398988.
Mulqueen, John T., “Product Analysis Review”,
Kleiman Steven R.
Viswanathan Srinivasan
Beausoliel Robert
Network Appliance Inc.
Puente Emerson
Swernofsky Law Group PC
LandOfFree
Recovery of file system data in file servers mirrored file... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Recovery of file system data in file servers mirrored file..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Recovery of file system data in file servers mirrored file... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3182147