Multiplex communications – Pathfinding or routing – Switching a message which includes an address header
Reexamination Certificate
2000-08-24
2004-06-01
Chin, Wellington (Department: 2664)
Multiplex communications
Pathfinding or routing
Switching a message which includes an address header
C370S349000, C370S473000, C370S474000, C709S236000, C709S238000, C714S748000, C714S749000
Reexamination Certificate
active
06744765
ABSTRACT:
BACKGROUND OF THE INVENTION
Most of today's distributed systems use shared-bus technology, e.g., peripheral component interconnect (PCI) cards, to connect computers to input/output (I/O) modules, e.g., video, graphics, Ethernet, small computer system interface (SCSI). For these distributed systems, there is a practical limit to the number of I/O modules that can be connected to the computer via the shared bus. There is also a limit to how far apart the I/O modules can be from the computer. Furthermore, all communications between the processor/memory complex of the computer and the I/O modules must pass through a single point of contention, the shared bus. All these factors, among others, pose limitations on the scalability, reliability, flexibility, and performance of the system. To address this problem, a group of computing industry leaders recently proposed an I/O architecture, called Infiniband
SM
, which defines a system area network for connecting various components of one or more computer systems. Examples of system area networks are known in the computing world, including High Performance Parallel Interface (HiPPI) and Fiber Channel technologies which are used to connect massively parallel processors to scalable storage servers and data vaults. U.S. Pat. No. 6,044,415 issued to Futral et al. discloses a virtual connection between an application program and an I/O device which is implemented as a system area network.
The Infiniband
SM
system area network consists of nodes which communicate through a channel-based, switched fabric. Each of the nodes could be a processor node, an I/O subsystem, a storage subsystem, or a router which connects to another network. The switched fabric is made of a collection of switches, routers, and links that connect a set of channel adapters. The channel adapters form an interface between the switched fabric and the nodes. The Infiniband
SM
system area network can be divided into subnets interconnected by routers. At this level, each Infiniband
SM
subnet is essentially a switched network. In general, switched networks are considered more scalable, i.e., more capable of growing to large number of nodes, than shared-media networks because of their ability to support many hosts at full speed. Infiniband
SM
is expected to provide a scalable performance of 500 Mbytes per second (4 Gbits per second) to 6 Gbytes per second (48 Gbits per second) per link.
In Infiniband
SM
, a client process has the ability to place a set of instructions that the hardware executes in a work queue. A client is the requesting program in a client/server relationship, and a process is an instance of a program running on a computer. Each process on a computer runs largely independently of other processes, and the operating system is responsible for making sure that resources, such as address space and CPU cycles, are allocated to all the current processes. The work queue holds instructions that cause data to be transferred between the client's memory and another process in one queue, called the send work queue, and instructions about where to place data that is received from another process in another queue, called the receive work queue. This other process is typically called a remote process, even if it is collocated on the same computer as the client process. The hardware executes the instructions in the order that they were placed in the work queue. For a send operation, messages are sent from the client process to the remote process in the form of a series of data units called packets. The sending hardware (sender) transmits the packets to a receiving hardware (receiver), where they can be accessed by the remote process. For operations such as remote direct memory access (RDMA) read operation, the remote process sends a reply message to the client process which contains the requested information.
Switches are used to route packets between the sender and the receiver. The switches typically route packets using either a datagram (or connectionless) network or a virtual-circuit (or connection-oriented) network. In a datagram network, each packet contains enough information, i.e., destination address, to enable any switch to decide how to get the packet to its destination. In a virtual-circuit network, a virtual connection is first set up between the source host and the destination host. This virtual connection may be set up by a network administrator. Alternatively, a host can send messages into the network to cause the state to be established. In a datagram-based network, a sequence of packets sent from a source host to a destination host may take different paths. Infiniband
SM
also supports a form of datagram-based network which is based upon explicit setup of switch routing tables by the subnet manager. In a virtual-circuit network, a sequence of packets sent from a source host to a receiver host takes the path established by the virtual circuit.
Infiniband
SM
provides reliable transport services between client and remote processes using a combination of packet sequence numbers (PSNs) and acknowledgement (ACK) messages. That is, each packet sent to the receiver is assigned a PSN, and the receiver sends an ACK message to the sender acknowledging receipt of the packet. A negative ACK (NAK) message is sent for dropped or lost packets. The ACK messages tell the sender what packets have been received at the remote end by providing the PSN of the received packet. A message is completed when all the outstanding packets for the message have been acknowledged. However, with just the returned PSNs, the sender has no effective way of knowing when the message has been completed. To determine when a message has been completed, the sender reads a descriptor in the client's memory space, for every returned PSN, to determine the size of the original message, i.e., the number of packets in the original message. Then the sender uses this information along with the PSN to determine whether the message has been completed. These extra reads of descriptors translate into additional system bus overhead on top of the data movement between the processor and memory. Schemes to minimize this overhead can significantly improve system performance.
SUMMARY OF THE INVENTION
In one aspect, the invention relates to a system of transmitting messages between a client process and a remote process which comprises a system area network providing a communications channel between the client process and the remote process. The system further includes a first channel adapter forming an interface between the client process and the communications channel. The first channel adapter is configured to receive a message from the client process, segment the message into a series of packets, assign a sequence number to each packet, and place the packets in order on the communications channel. The system further includes a second channel adapter forming an interface between the remote process and the communications channel. The second channel adapter is configured to receive packets from the communications channel and send at least one acknowledgement message to the first channel adapter in response to the received packets. The acknowledgement message has a packet sequence number field containing a packet sequence number and a payload containing a message sequence number. The message sequence number identifies a complete message last received at the second channel adapter, and the packet sequence number identifies a packet last received at the second channel adapter.
In some embodiments the client process has a work queue in which instructions to be executed by a communications interface are placed. In some embodiments the work queue comprises a send work queue in which messages to be sent to the remote process are placed, and the first channel adapter reads a message from the send work queue. In some embodiments the work queue further includes a receive work queue in which instructions about where to place a reply message received from the second channel adapter are placed.
In another aspect, t
Dearth Glenn A.
Ward Kenneth A.
Webber Thomas P.
Chin Wellington
Mais Mark A.
Osha & May L.L.P.
Sun Microsystems Inc.
LandOfFree
Mechanism for completing messages in memory does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Mechanism for completing messages in memory, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Mechanism for completing messages in memory will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3347176