Patent Application
20140226479
1. Field
Embodiments are generally directed to optimizing network traffic, and more specifically to optimizing network traffic using a parallel tori interconnect.
2. Background Art
A torus is a network topology for connecting processing nodes in a parallel computer network. A torus may be arranged in a field array of N dimensions, where processing nodes (also referred to as nodes) are connected to the nearest neighbors using links.
In a conventional torus network topology, a torus interconnect has a limited bandwidth. The bandwidth is limited because each host that connects to a subset of nodes in the torus receives a fraction of the bandwidth. Thus, connecting hosts to the torus fabric through more nodes, steals the bandwidth from other nodes in the torus and other hosts connected to these nodes.
Links propagate data traffic between nodes in the torus interconnect. When a link in a torus interconnect congests or fails, the data traffic between the nodes that use the affected link is rerouted. The rerouting affects traffic latency in the torus network. For example, the rerouted traffic may take longer to arrive to its destination node using the rerouted path. In another example, the traffic that was originally scheduled to flow through the rerouted path is also affected due to increased congestion caused by the rerouted data traffic.
A system and method for optimizing a flow of data traffic are provided. A plurality of tori are connected in a parallel tori interconnect. Each torus includes a plurality of nodes. The nodes in the torus are interconnected using links. A host in the network is connected to a subset of nodes where nodes in the subset are associated with different tori. The host transmits the packets to the parallel tori interconnect by selecting a node the subset of nodes. The packets are transmitted using links between from the node to the plurality of nodes in the torus, but not between the plurality of tori.
Further features and advantages of the embodiments, as well as the structure and operation of various embodiments, are described in detail below with reference to the accompanying drawings. It is noted that the embodiments are not limited to the specific embodiments described herein. Such embodiments are presented herein for illustrative purposes only. Additional embodiments will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein.
The accompanying drawings, which are incorporated herein and form part of the specification, illustrate the embodiments and, together with the description, further serve to explain the principles of the embodiments and to enable a person skilled in the pertinent art to make and use the embodiments. Various embodiments are described below with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout.
The embodiments will be described with reference to the accompanying drawings.
Generally, the drawing in which an element first appears is typically indicated by the leftmost digit(s) in the corresponding reference number.
In the detailed description that follows, references to “one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
The term “embodiments” does not require that all embodiments include the discussed feature, advantage or mode of operation. Alternate embodiments may be devised without departing from the scope of the disclosure, and well-known elements of the disclosure May not be described in detail or may be omitted so as not to obscure the relevant details. In addition, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. For example, as used herein, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise, It will be further understood that the terms “comprises,” “comprising,” “includes” and/or “including,” when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or groups thereof
A conventional torus interconnect is implemented as a single torus network topology. A single torus network topology has several limitations. First, it has a limited bandwidth. For example, each host connects to a certain number of nodes in the torus. Because each torus has a finite total bandwidth, each connection receives a fraction of a total available bandwidth in the torus. When data traffic between the nodes requires more bandwidth than the bandwidth allocated to them in the torus, two nodes may be connected to each other using multiple links. This solution, however, reduces the bandwidth in the torus that is available to other nodes. Additionally, when a data traffic source is allocated extra links to connect to multiple nodes in the torus, the bandwidth that other hosts can use to connect to the torus is also reduced.
Second, failure or congestion of a link between the nodes in a conventional torus interconnect adversely affects the network. For example, various types of data traffic flows between the nodes of the conventional torus interconnect. When one of the links or nodes either fails or congests, the failure or congestion may affect all data traffic flowing through the link. To remedy the congestion or failure, the conventional torus interconnect may redirect the data traffic through other nodes. This may cause delays in the overall network. This issue is particularly noticeable when the data traffic is associated with a quality of service (QoS) or a class of service (CoS) that cannot be met due to a link or node failure or congestion.
Creating redundant links is rudimentary way to remedy a link failure or congestion in the conventional torus interconnect. Although redundant links may decrease probability of the reduced quality of service, class of service, or congestion, redundant links may also decrease bandwidth in the conventional torus interconnect, as discussed above.
A parallel tori interconnect discussed below is a solution to the limitations described above.
In an embodiment, torus 102 represents a network. A network may be any network that carries data traffic and provides access to services and applications. A network may include, but is not limited to, a local area network (LAN), a metropolitan area network, and/or a wide area network (WAN), such as the Internet.
Nodes 104 are connectivity points in torus 102. In an embodiment, node 104 may be a computing device that is capable of sending, receiving, and forwarding data traffic over links 106. Example computing devices are described in detail in
In an embodiment, torus 102 may be assembled as a mesh. In a mesh, nodes 104 capture and disseminate own data as well as relay data traffic from other nodes 104.
Although torus 102 depicted in
In some embodiments, the links 106 between nodes 104 include one or more high-speed point-to-point serial communication links which utilize, for example, differential pair signaling between the connected processing nodes. For example, a bidirectional connection between nodes 104 can include one or more Peripheral Component Interconnect Express (PCIe) links or external PCIe links, such as a x1 PCIe link, a x4 PCIe link, a x8 PCIe link, or a x16 PCIe link, or a 10 Gigabit Ethernet (GbE) Attachment Unit Interface (XAUI) interface. In other embodiments, links 106 between nodes 104 may include Ethernet, Point-to-Point (PPP), High-Level Data Link Control (HDLC) protocol, and Advanced Data Communication Control Procedures (ADCCP) protocol interfaces, to name a few examples.
In an embodiment, host 108 uses sockets to connect to nodes 104 of different tori 102A-C. In an embodiment, a socket may include a socket address (such as an Internet Protocol or IP) address and a port number. In a parallel tori interconnect 101, nodes 104 from different tori 102 may connect to host 108 using the same socket address, but a different port number.
In an embodiment, each torus 102A, 102B and 102C is an independent, parallel, replica of others. When tori 102A-C in
In an embodiment, data traffic communicated between nodes 104 is segmented into packets. The packets are routed over a path between the source node and the destination node in one of tori 102 in parallel tori interconnect 101. In an embodiment, a source node is node 104 that connects to host 108 that transmits the packets into torus 102. In an embodiment, a destination node is node 104 that receives, stores and displays the data in the packet, but may not further propagate the packet. The path may include zero, one, or more than one intermediate nodes. In an embodiment, each node 104 includes an interface to the fabric interconnect that implements a link layer switch to route packets among the ports of the node connected to corresponding links of the fabric interconnect.
In an embodiment, host 108 connected to nodes 104 in different tori 102 selects a particular torus 102 to propagate data traffic. In one example, host 108 selects torus 102 based on a type of data traffic or preconfigured QoS requirements for different types of data. For instance, torus 102A may propagate data traffic having a “gold” QoS type, torus 102B may propagate data traffic having a “silver” QoS type, and torus 102C may propagate data traffic having a “bronze” QoS type, where the “gold”, “silver”, and “bronze” QoS types identify the upper bound of the guaranteed time that data traffic takes to arrive from a source node to a destination node. In another example, host 108 selects torus 102 based on congestion in torus 102. For example, if torus 102A experiences data traffic congestion, host 108 may transmit data traffic using torus 102B or 102C. Host 108 thus has control of distributing data traffic having a particular QoS across parallel tori interconnect 101, whereas nodes 104 within each torus 102 have control for propagating data traffic having a particular QoS within torus 102.
In another example, host 108 selects torus 102 based on a type of a CoS.
Example CoS may include a particular confidentiality group, a customer association, etc., that is represented in the data traffic. A type of CoS may be preconfigured within each CoS. In an embodiment, the type of CoS may be included in a data or voice protocols that is used to differentiate between different types of data traffic.
In another example, host 108 distributes data traffic across some or all parallel tori 102 according to a preconfigured algorithm in host 108. The algorithm may, for example, cause host 108 to monitor network congestion in each torus 102. In this embodiment, the effects of a node or link failure decrease because host 108 may re-route the data traffic to other parallel tori 102 based on traffic congestion or link failure in parallel tori interconnect 101.
When tori 102 are connected in parallel tori interconnect 101, the bandwidth of parallel tori interconnect 101 increases linearly with the number of tori 102. For example, the bandwidth of the network increases linearly by the bandwidth of each torus 102 added to parallel tori interconnect 101.
Scalability of software that manages parallel tori interconnect 101 is another advantage of parallel tori interconnect 101. For example, management software for parallel tori interconnect 101 may be scaled to manage each additional torus 102 added to parallel tori interconnect 101 and data traffic distribution to added torus 102. In an embodiment, when an additional torus 102 is added to parallel tori interconnect, the bandwidth between hosts 108 connected to nodes 104 in tori 102 prior to the addition, increases.
In an embodiment, the management software also manages QoS on each torus 102. In an embodiment, the management software executes on host 108 and distributes data traffic to parallel tori interconnect 101. In one example, the management software may distribute data traffic to each torus 102 according to QoS, as described above. In another example, the management software may distribute data traffic according to a security level. For example, data traffic associated with one security level may be distributed to one torus 102, and data traffic having another security level may be distributed to a different torus 102. This way, data traffic having different security levels is not transported over a single torus. Additionally, torus 102 that propagates data traffic having a particular security level can include additional security precautions. A person of ordinary skill in the art will appreciate that a security level may be set by an application or by a user using an application that sends or receives data.
In a further embodiment, hosts 108 may be restricted from sending data traffic to a particular torus 102 in parallel tori interconnect 101. For example, host 108 may be restricted to distributing data to a subset of tori 102 in parallel torus interconnect 101. One way to restrict the distribution of data is to connect host 108 to nodes 104 in the subset of tori (not shown). In another embodiment, host 108 may be physically connected to nodes 104 in parallel torus interconnect 101, but have the management software determine when to start and stop sending data to the connected nodes 104.
At operation 202, a host receives data traffic. For example, host 108 receives data traffic for distribution through parallel tori interconnect 101.
At operation 204, the host selects a node that receives the data traffic. For example, host 108 is connected to a subset of nodes 104 in parallel tori interconnect 101, where nodes 104 in the subset of nodes are associated with different tori 102. For instance, host 108 may be connected to node (0,0,0) in torus 102A, node (2,2,0) in torus 102B and node (2,0,0) in torus 102C. When host 108 receives the data traffic, as, for example, packets, host 108 selects node 104 from the subset of nodes to receive the data traffic. As discussed above, the selection may be based on the congestion in tori 102 in parallel tori interconnect 101, a type of QoS specified in the data traffic, or security level associated with the data traffic, to name a few examples. For instance, based on the above, host 108 may select node (0,0,0) in torus 102A, node (2,2,0) in torus 102B or node (2,0,0) in torus 102C.
At operation 206, the data traffic is propagated to the selected node. For example, host 108 propagates the packets to node 104 selected in operation 204.
Each FRU includes components disposed on a PCB, whereby the components are interconnected via metal layers of the PCB and provide the functionality of the node represented by the FRU. For example, the FRU 306 includes a PCB 312 implementing a processor 320 comprising one or more processor cores 322, one or more memory modules 324, such as DRAM dual inline memory modules (DIMMs), and a fabric interface device 326. Each FRU further includes a socket interface 330 that operates to connect the FRU to the interconnect 302 via the plug-in socket 304.
The interconnect 302 provides data communication paths between the plug-in sockets 304, such that the interconnect 302 operates to connect FRUs into rings and to connect the rings into a 2D- or 3D-torus network topology, such as the torus network 100B of
Various embodiments may be implemented by software, firmware, hardware, or a combination thereof.
Computer system 400 includes one or more processors, such as processor 410.
Processor 410 can be a special purpose or a general purpose processor. One example processor is a uses a central processing unit (“CPU”) to process data. A CPU is a processor which carries out instructions of computer programs or applications. For example, a CPU carries out instructions by performing arithmetical, logical and input/output operations. In an embodiment, a CPU performs control instructions that include decision making code of a computer program or an application, and delegates processing to other processors in the electronic device, such as a graphics processing unit (“GPU”). A GPU, is another example processor that is a specialized electronic circuit designed to rapidly process mathematically intensive applications on electronic devices. The GPU has a highly parallel structure that is efficient for parallel processing of large blocks of data, such as mathematically intensive data common to computer graphics applications, images and videos. The GPU may receive data for processing from a CPU or generate data for processing from previously processed data and operations. In an embodiment, the GPU is a hardware-based processor that uses hardware to process data in parallel.
Processor 410 is connected to a communication infrastructure 420 (for example, a bus or network).
Computer system 400 also includes a main memory 430, preferably random access memory (RAM), and may also include a secondary memory 440. Secondary memory 440 may include, for example, a hard disk drive 450, a removable storage drive 460, and/or a memory stick. Removable storage drive 460 may comprise a floppy disk drive, a magnetic tape drive, an optical disk drive, a flash memory, or the like. The removable storage drive 460 reads from and/or writes to a removable storage unit 470 in a well-known manner. Removable storage unit 470 may comprise a floppy disk, magnetic tape, optical disk, etc. which is read by and written to by removable storage drive 460. As will be appreciated by persons skilled in the relevant art(s), removable storage unit 470 includes a computer usable storage medium having stored therein computer software and/or data.
In alternative implementations, secondary memory 440 may include other similar means for allowing computer programs or other instructions to be loaded into computer system 400. Such means may include, for example, a removable storage unit 470 and an interface (not shown). Examples of such means may include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM, or PROM) and associated socket, and other removable storage units 470 and interfaces which allow software and data to be transferred from the removable storage unit 470 to computer system 400.
Computer system 400 may also include a communications and network interface 480. Communication and network interface 480 allows software and data to be transferred between computer system 400 and external devices. Communications and network interface 480 may include a modem, a communications port, a PCMCIA slot and card, or the like. Software and data transferred via communications and network interface 480 are in the form of signals which may be electronic, electromagnetic, optical, or other signals capable of being received by communication and network interface 480. These signals are provided to communication and network interface 480 via a communication path 485. Communication path 485 carries signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link or other communications channels.
The communication and network interface 480 allows the computer system 400 to communicate over communication networks or mediums such as LANs, WANTs the Internet, etc. The communication and network interface 480 may interface with remote sites or networks via wired or wireless connections.
In this document, the terms “computer program medium” and “computer usable medium” and “computer readable medium” are used to generally refer to media such as removable storage unit 470, removable storage drive 460, and a hard disk installed in hard disk drive 450. Signals carried over communication path 485 can also embody the logic described herein. Computer program medium, computer usable medium, and computer readable medium can also refer to memories, such as main memory 430 and secondary memory 440, which can be memory semiconductors (e.g. DRAMs, etc.). These computer program products are means for providing software to computer system 400.
Computer programs (also called computer control logic) are stored in main memory 430 and/or secondary memory 440. Computer programs may also be received via communication and network interface 480. Such computer programs, when executed, enable computer system 400 to implement embodiments as discussed herein. In particular, the computer programs, when executed, enable processor 410 to implement the processes of the embodiments, such as the steps in the methods illustrated by flowcharts discussed above. Accordingly, such computer programs represent controllers of the computer system 400. Where the embodiment is implemented using software, the software may be stored in a computer program product and loaded into computer system 400 using removable storage drive 460, interfaces, disk drive 450 or communication and network interface 480, for example.
The computer system 400 may also include input/output/display devices 490, such as keyboards, monitors, pointing devices, etc.
Embodiments can be accomplished, for example, through the use of general-programming languages (such as C or C++), hardware-description languages (HDL) including Verilog HDL, VHDL, Altera HDL (AHDL) and so on, or other available programming and/or schematic-capture tools (such as circuit-capture tools). The program code can be disposed in any known computer-readable medium including semiconductor, magnetic disk, or optical disk (such as CD-ROM, DVD-ROM). As such, the code can be transmitted over communication. networks including the Internet and internets. It is understood that the functions accomplished and/or structure provided by the systems and techniques described above can be represented in a core (such as a CPU core and/or a GPU core) that is embodied in program code and may be transformed to hardware as part of the production of integrated circuits.
The embodiments are also directed to computer program products comprising software stored on any computer-usable medium. Such software, when executed in one or more data processing devices, causes a data processing device(s) to operate as described herein or, as noted above, allows for the synthesis and/or manufacture of electronic devices (e.g., ASICs, or processors) to perform embodiments described herein. Embodiments employ any computer-usable or -readable medium, and any computer-usable or -readable storage medium known now or in the future. Examples of computer-usable or computer-readable mediums include, but are not limited to, primary storage devices (e.g., any type of random access memory), secondary storage devices (e.g., hard drives, floppy disks, CD ROMS, ZIP disks, tapes, magnetic storage devices, optical storage devices, MEMS, nano-technological storage devices, etc.), and communication mediums (e.g., wired and wireless communications networks, local area networks, wide area networks, intranets, etc.).
It is to be appreciated that the Detailed Description section, and not the Summary and Abstract sections, is intended to be used to interpret the claims. The Summary and Abstract sections may set forth one or more but not all exemplary embodiments as contemplated by the inventor(s), and thus, are not intended to limit the embodiments and the appended claims in any way,
The embodiments have been described above with the aid of functional building blocks illustrating the implementation of specified functions and relationships thereof. The boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternate boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed.
The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments that others can, by applying knowledge within the skill of the art, readily modify and/or adapt for various applications such specific embodiments, without undue experimentation, without departing from the general concept of the disclosure. Therefore, such adaptations and modifications are intended to be within the meaning and range of equivalents of the disclosed embodiments, based on the teaching and guidance presented herein. It is to be understood that the phraseology or terminology herein is for the purpose of description and. not of limitation, such that the terminology or phraseology of the present specification is to be interpreted by the skilled artisan in light of the teachings and guidance.
The breadth and scope of the embodiments should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.
