Error detection/correction and fault detection/recovery – Data processing system error or fault handling – Reliability and availability
Reexamination Certificate
1998-04-30
2001-06-12
Hayes, Gail (Department: 2131)
Error detection/correction and fault detection/recovery
Data processing system error or fault handling
Reliability and availability
C709S227000, C714S037000
Reexamination Certificate
active
06247139
ABSTRACT:
FIELD OF THE INVENTION
The present invention relates generally to techniques for increasing the availability of computer filesystems. More specifically, the present invention includes a method and apparatus for transparent failover of a filesystem in an environment where the filesystem is shared by a group of computers.
BACKGROUND OF THE INVENTION
Computer clusters are an increasingly popular alternative to more traditional computer architectures. A computer cluster is a collection of individual computers (known as nodes) that are interconnected to provide a single computing system. The use of a collection of nodes has a number of advantages over more traditional computer architectures. One easily appreciated advantage is the fact that nodes within a computer cluster may fail individually. As a result, in the event of a node failure, the majority of nodes within a computer cluster may survive in an operational state. This has made the use of computer clusters especially popular in environments where continuous availability is required.
Single system image (SSI) clusters are a special type of computer cluster. SSI clusters are configured to provide programs (and programmer's) with a unified environment in which the individual nodes cooperate to present a single computer system. Resources, such as filesystems, are made transparently available to all of the nodes included in an SSI cluster. As a result, programs in SSI clusters are provided with the same execution environment regardless of their physical location within the computer cluster. SSI clusters increase the effectiveness of computer clusters by allowing programs (and programmers) to ignore many of the details of cluster operation. Compared to other types of computer clusters, SSI clusters offer superior scaleablity (the ability to incrementally increase the power of the computing system), and manageability (the ability to easily configure and control the computing system). At the same time, SSI clusters retain the high availability of more traditional computer cluster types.
As the size of a computer cluster increases, so does the chance for failure among the cluster's nodes. Failure of a node has several undesirable effects. One easily appreciated effect is the performance degradation that results when the work previously performed by a failed node is redistributed to surviving nodes. Another undesirable effect is the potential loss of a resource, such as a filesystem, that is associated with a failed node.
Node loss can be especially serious in SSI clusters. This follows because resources are transparently shared within SSI clusters. Sharing of resources means that a single resource may be used by a large number of processes spread throughout an SSI cluster. If node failure causes the resource to become unavailable, each of these processes may be negatively impacted. Thus, a single node failure may impact many processes. Resource sharing also increases the likelihood that a process will access resources located on a number of different nodes. In so doing, the process becomes vulnerable to the failure of any of these nodes.
To ensure reliability, SSI clusters employ a number of different techniques. Failover is one of these techniques. To provide failover for a resource, the resource is associated with at least two nodes. The first of these nodes provides access to the resource during normal operation of the SSI cluster. The second node functions as a backup and provides access to the resource in the event that the first node fails. Failover, when properly implemented, greatly reduces the vulnerability of an SSI cluster to node failure.
In SSI clusters, filesystems are one of the most commonly shared resources. Thus, filesystem failover is especially important to the reliable operation of SSI clusters. Unfortunately, proper implementation of filesystem failover is a difficult task. This is particularly true in cases where filesystem performance is also a key consideration. For example, to increase performance of a shared filesystem, it is often necessary to aggressively cache the filesystem at each node where the filesystem is used. In cases where the filesystem fails over, it is imperative to maintain the consistency of the filesystem. Maintaining consistency during failover becomes increasingly problematic as caching becomes more aggressive. Thus, there is a need for techniques that balance the need to achieve high-performance filesystem operation and the need to provide failover protection.
SUMMARY OF THE INVENTION
An embodiment of the present invention includes a method and apparatus for filesystem failover in an SSI cluster. A representative environment for the present invention includes an SSI computer cluster. The SSI computer cluster includes a series of individual computer systems referred to as nodes. The nodes of the SSI computer cluster operate under control of UNIX® or UNIX-like operating systems.
Within the SSI cluster, one or more filesystems may be configured for failover protection. Each failover protected filesystem is located on a dual-ported disk (or other media that is accessible by more than one node). Two nodes are associated with each failover protected filesystem. The first node associated with a failover protected filesystem is the filesystem's active server node. The second node associated with a failover protected filesystem is the filesystem's standby server node.
Failover protected filesystems are mounted on their active server nodes as physical UNIX® filesystems. Processes do not, however, directly access failover protected filesystems using the physical UNIX® filesystems. Instead, processes access the mounted filesystems using a cluster filing environment (CFE). CFE, in turn, uses the physical UNIX® filesystem as needed. CFE is a distributed filesystem and includes a cluster filesystem (CFS), a cluster mount service (CMS) and a token manager.
CFS acts as a layer that is stacked onto the underlying physical UNIX® filesystems. Each active server node includes an instance of the CFS for each mounted filesystem. CFS instances are dynamically created on each node that uses a failover protected filesystem (a client node is a node that is not the active server node for a failover protected filesystem that uses the failover protected filesystem). Each CFS instance provides an interface to its associated failover protected filesystem. Coherency between the various instances of the CFS (on the client nodes or the active server nodes) is maintained through the use of the token manager. In this way, each CFS instance associated with a failover protected filesystem provides identical data and other filesystem attributes. The existence and location of each mounted filesystem is tracked by the CMS.
Processes (on the client nodes or the active server nodes) perform operations on failover protected filesystems exclusively by use of the CFS layer. The CFS layer monitors each operation that processes perform on failover protected filesystems. If an active server node fails during an operation, the CFS layer causes the process performing the operation to sleep in an interruptable state. When the failover protected filesystem on which the process was performing the operation later becomes available (i.e., when it is failed over to its standby server node), the CFS layer awakens the sleeping process and completes the operation.
The operational status of the nodes within the SSI cluster is monitored by a deamon process. If the active server node for a non-root failover protected filesystem fails, the deamon process notifies the failover protected filesystem's standby server node. In response, the standby server node carefully checks the integrity of the UNIX® filesystem associated with the failover protected filesystem. The standby server node then mounts the UNIX® filesystem associated with the failover protected filesystem. The existing CFS instance (originally located on the active server node) is then associated with the mounted filesystem on the standby server node. At this point
Byrne John L.
Chow William W.
Gertwagen John A.
Ramirez Laura L.
Walker Bruce J.
Compaq Computer Corp.
Fenwick & West LLP
Hayes Gail
Revak Christopher
LandOfFree
Filesystem failover in a single system image environment does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Filesystem failover in a single system image environment, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Filesystem failover in a single system image environment will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-2464331