Multiplex communications – Data flow congestion prevention or control – Control of data admission to the network
Reexamination Certificate
2000-01-18
2003-09-30
Kizou, Hassan (Department: 2662)
Multiplex communications
Data flow congestion prevention or control
Control of data admission to the network
C370S252000, C370S409000, C709S230000, C709S238000
Reexamination Certificate
active
06628615
ABSTRACT:
BACKGROUND OF THE INVENTION
1. Field of the Invention
This present invention relates generally to the field of network connected multiprocessor systems, and, more specifically, to a mechanism for improving the performance of data transmission through the network of such systems.
2. Discussion of the Prior Art
Network connected multiprocessor systems typically comprise nodes which communicate through a switching network. The nodes may be uni-processor workstations or bus based shared memory multiprocessor systems (SMP). A node may also be an I/O subsystem such a disk array which itself contains an I/O processor. In such systems, a variety of traffic may be communicated over the inter-connection network: shared memory coherence and data messages, TCP/IP packets, disk blocks, user level messaging, etc.. Each type of traffic relies on certain properties of the network to provide the type of service the producers and consumers of that traffic expect. In some cases latency is critical, such as with shared memory coherence traffic and with some types of user level messages. In other cases, throughput is more critical than latency, such as with disk accesses. In some cases, a quality of service guarantee in terms of latency or bandwidth is required. The challenge for the interconnection network in such systems is to provide appropriate characteristics for each data type. Except for the quality of service case, this typically involves balancing requirements of one data type against those of another. Various types of inter-connection networks have been devised to address the general problem of providing good latency and throughput for a variety of traffic types. Most of these techniques have been developed in the context of packet switched (as opposed to circuit switched) networks. In these networks, the original message to be transmitted is decomposed into two smaller units. At one level, a message is broken into packets which may be fixed or variable in length. At the next level, packets are broken into fixed sized ‘flits’. A flit is the fundamental data unit that can be interleaved at the lowest level of the network (i.e. the switching elements and the physical wires that connect them). The flit is also the level at which most techniques for enhancing network latency and throughput have been deployed.
The earliest packet switched networks were “store and forward” networks. In a store and forward network entire packets are passed from switching element to switching element. A subsequent packet is not transmitted until the entire packet in front of it completed transmission. A later enhancement to this basic approach was “wormhole routing”. With wormhole routing the notion of a flit was introduced. Now, instead of waiting for an entire packet to be received into a switching element before forwarding, the first flits of the packet could be transmitted to a down stream switching element even before the later flits have been received from the up stream switching element. In this way, a packet could be stretched across the entire network through a ‘wormhole route’. Wormhole routing significantly improves latency in lightly loaded networks, however it can severely degrade network throughput by blocking links that unrelated traffic could use, had not a wormhole route been in the way. A third type of network called “virtual cut-through” alleviated the blocking problem by providing enough buffering in the switching elements so that when a route is blocked an entire packet is guaranteed space so that it can be safely tucked entirely within the switching element. Of course, this guarantee comes at the expense of considerable space on the switching element, if it is to work efficiently.
A more recent development in packet switched networks for multi-processors is “virtual channels”. Each physical channel, i.e., wire link between switching elements, is conceptually partitioned amongst multiple ‘virtual’ channels. Each virtual channel includes a physical virtual channel buffer on the switching element. The virtual channels are multiplexed across common physical channels, but otherwise operate independently. A blocked packet flit on one virtual channel does not block packet flits on a different virtual channel over a common physical channel.
Virtual channel networks provide better network utilization and reduce average communication latency by allowing data on one virtual channel (or lane) to overtake data on a different virtual channel when there is contention downstream on one channel but not on another. Another desirable property is guaranteed ordering of transmissions on each channel and the ability to prioritize different data types. One factor that mitigates the improvement in network utilization is fragmentation of bandwidth on network links due to underutilized network flits. This can occur when data types assigned to different virtual channels are smaller than the flit. If these data types are communicated frequently, but not frequently enough to allow multiple of them to be packed into a flit, the flits become under utilized which can result in network under-utilization. Furthermore, there is a motivation to make flits large to increase the payload to overhead ratio, which only exacerbates the problem. Also, if the flit size is optimized for communication of large objects such as IP packets, the network may not be suitable for communication of smaller objects such as cache lines.
It would be highly desirable to provide a network interface scheme that improves utilization in virtual channel networks and provides greater flexibility in how different data types are handled by the network.
SUMMARY OF THE INVENTION
It is an object of the invention to provide a network interface scheme that improves utilization in virtual channel networks and provides greater flexibility in how different data types are handled by the network.
According to the invention, there is provided a second level of virtual channels at the network interface, and particularly, the designation of many second level channels which may share a single first level channel on the network. First level channels operate within the network at the switch level and are only used for network decongestion. In general, virtual channels also provide prioritization of different types of communication, which capability can become essential when, for example, one data type blocking another leads to deadlocks. However, in the design of the invention, prioritization is handled by the second level channels which operate at the network interfaces, not at the switch level. The network switching elements are oblivious to the existence of second level channels.
Thus, the provision of a two level virtual channels network interface scheme decouples the issue of network congestion from the issue of how different types of communication are to be handled (with respect to the role of virtual channels in the network). The nodes of the system may be randomly assigned to a first level virtual channel and each node defines independently its own second level virtual channels. Alternatively, nodes may be assigned to virtual channels on the basis of how the system is partitioned, which reduces network interference between partitions. The number of second level channels and how they are managed may differ from node to node, as long as all the nodes that communicate with one another maintain a consistent policy amongst themselves. Each message type that a node recognizes is assigned to its own second level virtual channel. The network interface is responsible for packing the different message types from the second level channels onto its assigned first level channel. The ability to pack more than one message type on a single first level channel allows for more efficient use of network flits, which is how two level virtual channels can provide higher network utilization than single level virtual channels.
According to this scheme, three (3) message classes are supported by the mechanism: Latency Sensitive, Bandwidth Sensitive and Bi-Modal. Latency sensitive message
Joseph Douglas J.
Michael Maged M.
Nanda Ashwini
Jennings Derek S.
Kizou Hassan
Qureshi Afsar M.
Scully Scott Murphy & Presser
LandOfFree
Two level virtual channels does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Two level virtual channels, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Two level virtual channels will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3092742