The present invention relates to a computing system in which resource blades are interconnected by a network.
A computing system of a data center in related art is constructed by using a server. Various resources (CPU, memory, storage, accelerator) necessary for a calculation task are mounted on the server little by little. These resources are tightly integrated by a bus. A computing system in which a plurality of servers are prepared and interconnected by a network is a currently typical computing system of a data center.
Such server-centric architectures have been mainstream for many decades. In contrast, recently, a disaggregated data center (DDC) has been proposed in which resources such as a central processing unit (CPU) and a memory are used as resource blades for each function of the resources, and the resource blades are interconnected by a network fabric (see Non Patent Literature 1).
In the DDC in which the resources are separated, each resource technology can easily independently advance, and a period until adoption can be shortened by avoiding a burden such as integration and rework of mother board design.
In the disaggregated computing technology as illustrated in the example of
Embodiments of the present invention have been made to solve the above problems, and an object of embodiments of the present invention is to provide a computing system capable of reducing a possibility of occurrence of congestion in a network.
A computing system of embodiments of the present invention includes: a plurality of top-of-rack switches that connect subordinate resource blades; a spine switch that connects the plurality of top-of-rack switches; and a plurality of PIN blades arranged between the top-of-rack switches and subordinate resource blades thereof, each of the PIN blades includes a plurality of processing blocks, each of the processing blocks includes a first packet reception unit that receives a packet from the subordinate resource blade; a first packet transmission unit that transmits a packet to the subordinate resource blade; a second packet reception unit that receives a packet from the upper top-of-rack switch; a second packet transmission unit that transmits a packet to the upper top-of-rack switch, a functional unit that performs predetermined processing on data included in a received packet; and a flow ID switch that determines a transfer destination of the data included in the packets received by the first and second packet reception units or the data processed by the functional unit, from the first and second packet transmission units and the functional unit and transfer the data to the transfer destination.
In addition, in one configuration example of the computing system of embodiments of the present invention, a plurality of processing blocks in each of the PIN blades are connected to each other, and the flow ID switch of each of the processing blocks receives data from another processing block in the same PIN blade in addition to the data included in the packets received by the first and second packet reception units and the data processed by the functional unit, determines a transfer destination of the received data from other processing blocks in the same PIN blade as the first and second packet transmission units and the functional unit and transfers the data to the transfer destination.
Further, in one configuration example of the computing system of embodiments of the present invention, the first packet reception unit restores data from the packet received from the resource blade, adds a flow ID corresponding to transmission destination information included in the received packet to the data and transfers the data to the flow ID switch, the second packet reception unit restores data from the packet received from the top-of-rack switch, adds a flow ID corresponding to the transmission destination information included in the received packet to the data and transfers the data to the flow ID switch, the flow ID switch determines a transfer destination of the data on the basis of the flow ID added to the received data, the first packet transmission unit packetizes the data received from the flow ID switch, sets the transmission destination information corresponding to the flow ID added to the received data in the packet and transmits the packet to the resource blade, and the second packet transmission unit packetizes the data received from the flow ID switch, sets the transmission destination information corresponding to the flow ID added to the received data in the packet and transmits the packets to the top-of-rack switch.
In addition, in one configuration example of the computing system of embodiments of the present invention, the functional unit performs processing of any of compression of data, decompression of compressed data, resizing of an image, and channel division of an image.
According to embodiments of the present invention, PIN blades are provided between top-of-rack switches and subordinate resource blades thereof, and first and second packet reception units, first and second packet transmission units, functional units, and a flow ID switch are provided in each processing block of the PIN blades. In embodiments of the present invention, data is processed and transmitted by the functional units of the PIN blades in communication between resource blades, so that load of a network can be reduced, and a possibility that congestion occurs in the network can be reduced.
Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
As described above, examples of the resource blades include CPU blades 100-1, 100-2, accelerator blades 101-1, 101-2, and storage blades 102-1, 102-2.
The PIN blades 105-1, 105-2 apply data size reduction processing such as encoding/decoding or compression/restoration to the input data. In addition, the PIN blades 105-1, 105-2 perform small processing, which has been required to be processed by the CPU in related art, at a network line rate instead of the CPU. In addition, the PIN blades 105-1, 105-2 have a simple circuit switching function.
Each of the processing blocks 1-1 to 1-3 has a plurality of network physical ports, and a port on one side is connected to the resource blade and a port on the other side is connected to the ToR switch 103-1. The processing blocks 1-1 to 1-3 are connected by a network or a bus and can communicate with each other.
As illustrated in
The packet reception units 10 of the processing blocks 1-1 and 1-2 receive packets from the connected accelerator blades 101-1a, 101-1b, 101-1c, and 101-1d. In the packet reception units 10, tables 200 in which a transmission destination IP address, a transmission destination port number, and a corresponding flow ID are registered in advance are set. The flow ID is an identifier given to the data to route the data in the PIN blade 105-1.
The packet reception units 10 restore original data from payloads of one or a plurality of packets received from the accelerator blades 101-1a, 101-1b, 101-1c, and 101-1d. The header of the packet includes a transmission destination IP address and a transmission destination port number as transmission destination information. The packet reception units 10 acquire the flow ID corresponding to the transmission destination IP address and the transmission destination port number included in the received packets from the tables 200, add the flow ID to the restored data, and transfer the data to the flow ID switches 12.
On the other hand, the packet reception units 11 of the processing blocks 1-1 and 1-2 receive packets from the connected ToR switch 103-1. In the packet reception units 11, tables 201 in which a transmission destination IP address, a transmission destination port number, and a corresponding flow ID are registered in advance are set.
The packet reception units 11 restore the original data from the payloads of one or a plurality of packets received from the TOR switch 103-1. The packet reception units 11 acquire the flow ID corresponding to the transmission destination IP address and the transmission destination port number included in the received packets from the tables 201, add the flow ID to the restored data, and transfer the data to the flow ID switches 12.
Tables 300 in which a flow ID and information of a corresponding transfer destination are registered in advance are set in the flow ID switches 12 of the processing blocks 1-1 and 1-2. If data is received from the packet reception units 10 and 11 or the flow ID switches 12 and 15 of other processing blocks in the same PIN blade, the flow ID switches 12 acquire information of the transfer destination corresponding to the flow ID added to the data from the tables 300 and transfer the data to the functional units 13 and 14 of the transfer destination specified by the information or the flow ID switches 12 and 15 of the processing blocks of the transfer destination.
The functional units 13 and 14 of the processing blocks 1-1 and 1-2 perform predetermined processing on the data from the flow ID switches 12. Examples of the processing to be performed by the functional units 13 include encoding, decoding, and compression of data, and decompression of compressed data. Examples of the processing to be performed by the functional units 14 include resizing of an image and channel division.
Tables 301 in which a flow ID and information of a corresponding transfer destination are registered in advance are set in the flow ID switches 15 of the processing blocks 1-1 and 1-2. If data is received from the flow ID switches 12, the functional units 13 and 14, or the flow ID switches 12 and 15 of other processing blocks in the same PIN blade, the flow ID switches 15 acquire information of the transfer destination corresponding to the flow ID added to the data from the tables 301 and transfer the data to the packet transmission units 16 and 17 of the transfer destination specified by the information or the flow ID switches 12 and 15 of the processing blocks of the transfer destination.
The packet transmission units 16 of the processing blocks 1-1 and 1-2 transmit packets to the connected accelerator blades 101-1a, 101-1b, 101-1c, and 101-1d. In the packet transmission units 16, tables 202 in which a flow ID and a corresponding transmission destination IP address and transmission destination port number are registered in advance are set.
If data is received from the flow ID switches 15, the packet transmission units 16 packetize the data and acquires the transmission destination IP address and the transmission destination port number corresponding to the flow ID added to the data from the tables 202. The packet transmission units 16 set a transmission destination IP address and a transmission destination port number in the headers of the created packets and transmit the packets to the accelerator blades 101-1a, 101-1b, 101-1c, and 101-1d.
The packet transmission units 17 of the processing blocks 1-1 and 1-2 transmit packets to the connected ToR switch 103-1. In the packet transmission units 17, tables 203 in which a flow ID and a corresponding transmission destination IP address and transmission destination port number are registered in advance are set.
If data is received from the flow ID switches 15, the packet transmission units 17 packetize the data and acquire the transmission destination IP address and the transmission destination port number corresponding to the flow ID added to the data from the tables 203. The packet transmission units 17 set the transmission destination IP address and the transmission destination port number in the headers of the created packets and transmit the packets to the ToR switch 103-1.
Although the PIN blade 105-1 is described as an example in
Next, a specific example of operation of the PIN blades 105-1, 105-2 of the present embodiment will be described.
First, an example of a case where data is transferred from the accelerator blade 101-1 to the accelerator blade 101-2 in
The packet reception unit 10 of the processing block in the PIN blade 105-1 restores the original data from the payload of the packet received from the accelerator blade 101-1. The packet reception unit 10 acquires the flow ID corresponding to the transmission destination IP address and the transmission destination port number included in the packet from the table 200, adds the flow ID to the restored data, and transfers the data to the flow ID switch 12.
If the data is received from the packet reception unit 10, the flow ID switch 12 of the processing block in the PIN blade 105-1 acquires information of the transfer destination corresponding to the flow ID added to the data from the table 300 and transfers the data to the transfer destination designated by the information. Here, it is assumed that the transfer destination is the functional unit 13.
If the data is received from the flow ID switch 12, the functional unit 13 of the processing block in the PIN blade 105-1 compresses the data and transfers the compressed data to the flow ID switch 15.
If the data is received from the functional unit 13, the flow ID switch 15 of the processing block in the PIN blade 105-1 acquires information of the transfer destination corresponding to the flow ID added to the data from the table 301 and transfers the data to the transfer destination designated by the information. Here, it is assumed that the transfer destination is the packet transmission unit 17.
If the data is received from the flow ID switch 15, the packet transmission unit 17 of the processing block in the PIN blade 105-1 packetizes the data and acquires the transmission destination IP address and the transmission destination port number corresponding to the flow ID added to the data from the table 203. The packet transmission unit 17 sets the transmission destination IP address and the transmission destination port number in the header of the created packet and transmits the packet to the ToR switch 103-1 via the network.
If the packet is received from the PIN blade 105-1, the ToR switch 103-1 refers to the transmission destination IP address set in the packet header and transmits the packet to the spine switch 104 via the network.
If the packet is received from the TOR switch 103-1, the spine switch 104 refers to the transmission destination IP address set in the packet header and transmits the packet to the ToR switch 103-2 via the network.
If the packet is received from the spine switch 104, the TOR switch 103-2 refers to the transmission destination IP address set in the packet header and transmits the packet to the PIN blade 105-2 via the network.
The packet reception unit 11 of the processing block in the PIN blade 105-2 restores the original data from the payload of the packet received from the TOR switch 103-2. The data restored here is the data compressed by the PIN blade 105-1. The packet reception units 11 acquire the flow ID corresponding to the transmission destination IP address and the transmission destination port number included in the received packets from the tables 201, add the flow ID to the restored data, and transfer the data to the flow ID switches 12.
If the data is received from the packet reception unit 11, the flow ID switch 12 of the processing block in the PIN blade 105-2 acquires information of the transfer destination corresponding to the flow ID added to the data from the table 300 and transfers the data to the transfer destination designated by the information. Here, it is assumed that the transfer destination is the functional unit 13.
If the data is received from the flow ID switch 12, the functional unit 13 of the processing block in the PIN blade 105-2 decompresses the compressed data to restore the original data and transfers the data to the flow ID switch 15.
If the data is received from the functional unit 13, the flow ID switch 15 of the processing block in the PIN blade 105-2 acquires information of the transfer destination corresponding to the flow ID added to the data from the table 301 and transfers the data to the transfer destination designated by the information. Here, it is assumed that the transfer destination is the packet transmission unit 16.
If the data is received from the flow ID switch 15, the packet transmission unit 16 of the processing block in the PIN blade 105-2 packetizes the data and acquires the transmission destination IP address and the transmission destination port number corresponding to the flow ID added to the data from the table 202. The packet transmission unit 16 sets a transmission destination IP address and a transmission destination port number in the header of the created packet and transmits the packet to the accelerator blade 101-2.
As described above, in the present embodiment, the data is compressed and transmitted in a route from the PIN blade 105-1 to the PIN blade 105-2 via the ToR switch 103-1, the spine switch 104, and the ToR switch 103-2, so that it is possible to reduce load of the network and reduce a possibility of congestion in the network.
Note that data size reduction is also effective in communication between resource blades connected to the same PIN blade. In addition, in a case of communication between the resource blades connected to the same PIN blade, communication between the resource blades can be implemented without passing through the TOR switch by a circuit switching function of the PIN blade.
For example, in a case where data is transferred from the accelerator blade 101-1a to the accelerator blade 101-1c in
If the data is received from the packet reception unit 10, the flow ID switch 12 of the processing block 1-1 in the PIN blade 105-1 acquires information of the transfer destination corresponding to the flow ID added to the data from the table 300 and transfers the data to the transfer destination designated by the information. Here, it is assumed that the transfer destination is the functional unit 13.
If the data is received from the flow ID switch 12, the functional unit 13 of the processing block 1-1 in the PIN blade 105-1 compresses the data and transfers the compressed data to the flow ID switch 15.
If the data is received from the functional unit 13, the flow ID switch 15 of the processing block 1-1 in the PIN blade 105-1 acquires information of the transfer destination corresponding to the flow ID added to the data from the table 301 and transfers the data to the transfer destination designated by this information. Here, it is assumed that the transfer destination is the flow ID switch 12 of the processing block 1-2 in the PIN blade 105-1.
If the data is received from the processing block 1-2 in the PIN blade 105-1, the flow ID switch 12 of the processing block 1-1 acquires information of the transfer destination corresponding to the flow ID added to the data from the table 300 and transfers the data to the transfer destination designated by the information. Here, it is assumed that the transfer destination is the functional unit 13.
If the data is received from the flow ID switch 12, the functional unit 13 of the processing block 1-2 in the PIN blade 105-1 decompresses the compressed data to restore the original data and transfers the data to the flow ID switch 15.
If the data is received from the functional unit 13, the flow ID switch 15 of the processing block 1-2 in the PIN blade 105-1 acquires information of the transfer destination corresponding to the flow ID added to the data from the table 301 and transfers the data to the transfer destination designated by this information. Here, it is assumed that the transfer destination is the packet transmission unit 16.
If the data is received from the flow ID switch 15, the packet transmission unit 16 of the processing block 1-2 in the PIN blade 105-1 packetizes the data and acquires the transmission destination IP address and the transmission destination port number corresponding to the flow ID added to the data from the table 202. The packet transmission unit 16 sets the transmission destination IP address and the transmission destination port number in the header of the created packet and transmits the packet to the accelerator blade 101-1C.
In the related art, it is known that fluctuation of a communication delay greatly affects quality of service. A general ToR switch performs packet-switched communication, and thus, a packet queue is generated, and a delay time fluctuates.
On the other hand, in the present embodiment, the delay time can be made constant by providing the PIN blade with a circuit switching function. In other words, in a case of communication between the resource blades connected to the same PIN blade, a direct path can be created between the resource blades connected to the same PIN blade by transmitting and receiving data between the flow ID switches without using the TOR switch.
Next, an example of a case where image processing is required in a case where image data is transferred from the accelerator blade 101-1 to the accelerator blade 101-2 in
The packet reception unit 10 of the processing block in the PIN blade 105-1 restores the original image data from the payload of the packet received from the accelerator blade 101-1. The packet reception unit 10 adds the flow ID to the restored image data and transfers the image data to the flow ID switch 12.
If the image data is received from the packet reception unit 10, the flow ID switch 12 of the processing block in the PIN blade 105-1 acquires information of the transfer destination corresponding to the flow ID added to the image data from the table 300 and transfers the image data to the transfer destination designated by this information. Here, it is assumed that the transfer destination is the functional unit 14.
If the image data is received from the flow ID switch 12, the functional unit 14 of the processing block in the PIN blade 105-1 performs predetermined processing on the image data and transfers the image data to the flow ID switch 15. Examples of the processing to be performed by the functional units 14 include resizing of an image and channel division.
If the image data is received from the functional unit 14, the flow ID switch 15 of the processing block in the PIN blade 105-1 acquires information of the transfer destination corresponding to the flow ID added to the image data from the table 301 and transfers the image data to the transfer destination designated by this information. Here, it is assumed that the transfer destination is the packet transmission unit 17.
If the image data is received from the flow ID switch 15, the packet transmission unit 17 of the processing block in the PIN blade 105-1 packetizes the image data and acquires the transmission destination IP address and the transmission destination port number corresponding to the flow ID added to the image data from the table 203. The packet transmission unit 17 sets the transmission destination IP address and the transmission destination port number in the header of the created packet and transmits the packet to the TOR switch 103-1 via the network.
The operation of the ToR switch 103-1, the spine switch 104, and the ToR switch 103-2 has been described above.
The packet reception unit 11 of the processing block in the PIN blade 105-2 restores the image data from the payload of the packet received from the ToR switch 103-2. The packet reception unit 11 adds the flow ID to the restored image data and transfers the image data to the flow ID switch 12.
If the image data is received from the packet reception unit 11, the flow ID switch 12 of the processing block in the PIN blade 105-2 acquires information of the transfer destination corresponding to the flow ID added to the image data from the table 300 and transfers the image data to the transfer destination designated by this information. Here, it is assumed that the transfer destination is the flow ID switch 15.
If the image data is received from the flow ID switch 12, the flow ID switch 15 of the processing block in the PIN blade 105-2 acquires information of the transfer destination corresponding to the flow ID added to the image data from the table 301 and transfers the image data to the transfer destination designated by this information. Here, it is assumed that the transfer destination is the packet transmission unit 16.
If the image data is received from the flow ID switch 15, the packet transmission unit 16 of the processing block in the PIN blade 105-2 packetizes the image data and acquires the transmission destination IP address and the transmission destination port number corresponding to the flow ID added to the image data from the table 202. The packet transmission unit 16 sets a transmission destination IP address and a transmission destination port number in the header of the created packet and transmits the packet to the accelerator blade 101-2.
As described above, in the present embodiment, the image data is processed by the PIN blade 105-1 in a route from the accelerator blades 101-1 to the accelerator blade 101-2 via the PIN blade 105-1, the TOR switch 103-1, the spine switch 104, the TOR switch 103-2, and the PIN blade 105-2.
In related art, small processing such as preprocessing and post-processing is generally performed by a CPU not by an accelerator. For example, image processing is generally performed by a graphics processing unit (GPU), but there is a case where resizing of an image, channel division, and the like, are performed by a CPU. In such a case, access to a CPU blade occurs in the DDC, which causes access tightness. In the present embodiment, the PIN blade takes over the processing instead of the CPU, so that the access to the CPU blade can be reduced, and the load of the network can be reduced.
Offloading of processing is also effective for communication between resource blades connected to the same PIN blade. For example, in a case where data is transferred from the accelerator blade 101-1a to the accelerator blade 101-1c in
If the image data is received from the packet reception unit 10, the flow ID switch 12 of the processing block 1-1 in the PIN blade 105-1 acquires information of the transfer destination corresponding to the flow ID added to the image data from the table 300 and transfers the image data to the transfer destination designated by the information. Here, it is assumed that the transfer destination is the functional unit 14.
If the image data is received from the flow ID switch 12, the functional unit 14 of the processing block 1-1 in the PIN blade 105-1 performs predetermined processing on the image data and transfers the image data to the flow ID switch 15.
If the image data is received from the functional unit 14, the flow ID switch 15 of the processing block 1-1 in the PIN blade 105-1 acquires the information of the transfer destination corresponding to the flow ID added to the image data from the table 301 and transfers the data to the transfer destination designated by this information. Here, it is assumed that the transfer destination is the flow ID switch 15 of the processing block 1-2 in the PIN blade 105-1.
If the image data is received from the processing block 1-2 in the PIN blade 105-1, the flow ID switch 15 of the processing block 1-1 acquires information of the transfer destination corresponding to the flow ID added to the image data from the table 301 and transfers the image data to the transfer destination designated by the information. Here, it is assumed that the transfer destination is the packet transmission unit 16.
If the image data is received from the flow ID switch 15, the packet transmission unit 16 of the processing block 1-2 in the PIN blade 105-1 packetizes the image data and acquires the transmission destination IP address and the transmission destination port number corresponding to the flow ID added to the image data from the table 202. The packet transmission unit 16 sets the transmission destination IP address and the transmission destination port number in the header of the created packet and transmits the packet to the accelerator blade 101-1c.
Embodiments of the present invention can be applied to a disaggregated computing technology in which resource blades are interconnected by a network.
This application is a national phase entry of PCT Application No. PCT/JP2021/044889, filed on Dec. 7, 2021, which application is hereby incorporated herein by reference.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/JP2021/044889 | 12/7/2021 | WO |