The present disclosure relates generally to optimizing Infiniband ports. More specifically, the disclosure relates to multiple single or double data rate Infiniband ports being time division multiplexed into a single double or quad data rate Infiniband port to optimize the use of the higher bandwidth provided by the higher rate port.
Generally used in high-performance computing systems, Infiniband is a bidirectional point-to-point serial link often used to connect processors to storage devices and other high speed peripherals. Infiniband signals currently have a connection signal rate of 2.5 gigabit per second for single data rate (SDR), 5 gigabit per second for double data rate (DDR), and 10 gigabit per second for quad data rate (QDR), and are encoded with the industry standard 8B/10B encoding (every 10 bits sent carries 8 bits of data). Thus, Infiniband ports carry 2, 4, and 8 gigabits per second of useful data for, respectfully, single, double, and quad data rates.
The Infiniband architecture allows for the design of networking switches with single data rate, double data rate, and quad data rate ports. Ports are capable of detecting the capabilities of the attached devices and automatically switching between rates. Currently, when utilizing a higher data rate port at a lower rate, the entire port is consumed. This is inefficient, as the excess bandwidth within the switch cannot be utilized.
A method in particular embodiments may include receiving one or more Infiniband signal streams, de-serializing the received one or more Infiniband signal streams to generate a corresponding one or more parallel data streams, de-serializing further including selectively adding or deleting a control symbol associated with a quiescent period of the one or more received signal streams, time-division multiplexing the generated one or more parallel data streams, and serializing the time division multiplexed one or more parallel data streams.
An apparatus in particular embodiments may include an input Infiniband network communications interface configured to receive one or more Infiniband signal streams, a processing and control unit operatively coupled to the input Infiniband network communication interface, the processing and control unit configured to de-serialize the received one or more Infiniband signal streams to generate a corresponding one or more parallel data streams, where de-serializing includes selectively adding or deleting a control symbol associated with a quiescent period of the one or more received signal streams, to time-division multiplex the generated one or more parallel data streams, and to serialize the time division multiplexed one or more parallel data streams, and an output Infiniband network communication interface operatively coupled to the processing and control unit configured to output one or more of the serialized parallel data streams.
Referring to
As described in further detail below, in particular embodiments, data streams from multiple lower rate Infiniband ports, such as single data rate or double data rate Infiniband ports, may be time division multiplexed (TDM) to aggregate the two or more data streams onto a single higher rate Infiniband port, such as a double data rate or quad data rate Infiniband port. In particular embodiments, the data is time division multiplexed into the higher data rate port at an interval smaller than the frame length. Time division multiplexing the data stream at an interval smaller than the frame length in particular embodiments allows low latency multiplexing as the entire frame does not need to be buffered before it is transmitted on the higher data rate port. In particular embodiments, combining multiple data streams may be performed at the physical link layer to avoid complicated decoding of the signal.
In particular embodiments, a predetermined control symbol, herein referred to as a K.28.6 symbol, of the 10B code of the Infiniband signal may be introduced to compensate for obstacles presented by the quiescent periods during initialization. The K.28.6 symbol is a predetermined control symbol used in standard 8B/10B encoding, which represents a sequence of bits, but does not have a corresponding 8 b data byte. For example, in particular embodiments, the K.28.6 control symbols may be used to denote the quiescent periods where no signal is present on the data transmission wire, and further the K.28.6 control symbols in particular embodiments may be used to pad the stream when necessary.
Still referring to
Still referring to
Still referring to
In particular embodiments, the K.28.6 control symbol may include an encoded skip signal to fill the quiescent periods of the data stream. In this manner, in particular embodiments, automatic conversion of quiescent periods of signals to a corresponding quiescent state may be avoided, thus preventing the possibility of loss or corruption of data in the signals. For example, if the input lower data rate signal is two SDR port signals, and the output higher data rate signal is QDR which has a data rate four times that of a single SDR, the K.28.6 control symbol may be used to pad the two SDR signals up to a data rate sufficiently high to fill the QDR data stream.
Still referring to
Accordingly, in particular embodiments, if the input data streams each include SDR Infiniband signals, and the desired output data rate is a QDR Infiniband data rate, the two SDR Infiniband signals are each padded with respective K.28.6 control symbols to increase the data rate to the equivalent of half the QDR rate. That is, each of the 2.5 Gbps SDR signals are padded with the K.28.6 control signals to result in 5 Gbps signals, which is half the rate of a 10 Gbps QDR rate. Once combined via time division multiplexing, the output data stream includes a 10B symbol from the first data stream, followed by a 10B symbol from the second data stream, followed by two K.28.6 control symbols (one K.28.6 control symbol from each of the two padded SDR signals), followed by another 10B symbol from the first data stream, and another 10B symbol from the second data stream, and so on until the entire combined signal was composed. In this manner, the resulting data rate is at the 10 Gbps quad data rate, and is comprised of two SDR signals padded with the K.28.6 control symbols.
In yet another embodiment, in the case where the two lower data rate input signals are DDR data streams at 5 Gbps, and the desired output higher data rate signal is QDR at 10 Gbps, the combined data rate of the two input signals is already that of a QDR signal, and thus padding by the K.28.6 control symbols is not necessary, and the output QDR data stream may comprise of alternating symbols from the first DDR signal and the second DDR signal.
Referring still to
Referring back to
Still referring to
Referring yet still to
Still referring to
Referring again to
Still referring to
Referring again to
In the manner described above, in particular embodiments, multiple single or double data rate Infiniband ports may be time division multiplexed into a corresponding single double or quad data rate Infiniband port to optimize the use of the higher bandwidth provided by the higher rate port. Time division multiplexing the stream at an interval smaller than the frame length allows low latency multiplexing as the entire frame does not need to be buffered before it is transmitted on the higher data rate port.
Accordingly, in particular embodiments, multiple lower rate Infiniband ports, such as single data rate or double data rate Infiniband ports, may be time division multiplexed to aggregate the two or more data streams onto a single higher rate Infiniband port, such as a double data rate or quad data rate Infiniband port. The data is time division multiplexed into the higher data rate port at an interval smaller than the frame length. Time division multiplexing the stream at an interval smaller than the frame length provides for low latency multiplexing as the entire frame does not need to be buffered before it is transmitted on the higher data rate port. Further, the combining of the two streams may be performed at the physical link layer, avoiding any complication associated with decoding the signals.
Additionally, a control symbol (K.28.6) of the 10B code of the Infiniband signal may be introduced to compensate for the obstacles presented by the quiescent periods during initialization that Infiniband uses. These control symbols are used to denote the quiescent periods where no signal is present on the wire, where if these periods were not encoded, the time division multiplexed combined version of the signals will have symbols directly followed by quiescent periods. Moreover, in particular embodiments, the K.28.6 control symbols may be used to pad the data stream as desired.
In one embodiment of the present disclosure, a method may comprise receiving one or more Infiniband signal streams, de-serializing the received one or more Infiniband signal streams to generate a corresponding one or more parallel data streams, de-serializing further including selectively adding or deleting a control symbol associated with a quiescent period of the one or more received signal streams, time-division multiplexing the generated one or more parallel data streams, and serializing the time division multiplexed one or more parallel data streams.
In one aspect the serialized time division multiplexed one or more parallel data streams may include an output signal stream having a data rate higher than the data rate of the received one or more Infiniband signal streams.
The received one or more Infiniband signal streams may be double data rate Infiniband signal streams.
The received one or more Infiniband signal streams may be single data rate Infiniband signal streams.
Further, the control symbol may be padded to one or more of the received one or more Infiniband signal streams, wherein the combined data rate of the received one or more Infiniband signal streams after being padded with the control symbol, is substantially equal to the data rate of the serialized multiplexed one or more parallel data streams.
In another aspect, the time division multiplexed one or more parallel data streams may include two or more output signal streams, wherein the two or more output signal streams each have a data rate lower than the data rate of the received one or more Infiniband signal streams.
The received one or more Infiniband signal streams may be a quad data rate Infiniband signal stream.
The received one or more Infiniband signal streams may be a double data rate Infiniband signal stream.
A further aspect may comprise detecting and selectively removing the control symbol from the time division multiplexed one or more parallel data streams.
In yet a further aspect, when the time division multiplexed one or more parallel data streams are comprised entirely of the control symbols, the received one or more Infiniband signal streams may be an idle stream.
Another aspect may comprise transmitting the serialized time division multiplexed one or more parallel data streams.
In another embodiment, an apparatus may comprise an input Infiniband network communications interface configured to receive one or more Infiniband signal streams, a processing and control unit operatively coupled to the input Infiniband network communication interface, the processing and control unit configured to de-serialize the received one or more Infiniband signal streams to generate a corresponding one or more parallel data streams, wherein de-serializing includes selectively adding or deleting a control symbol associated with a quiescent period of the one or more received signal streams, to time-division multiplex the generated one or more parallel data streams, and to serialize the time division multiplexed one or more parallel data streams, and an output Infiniband network communication interface operatively coupled to the processing and control unit configured to output one or more of the serialized parallel data streams.
In one aspect, the serialized time division multiplexed one or more parallel data streams may include an output signal stream having a data rate higher than the data rate of the received one or more Infiniband signal streams.
Further, the processing and control unit may be configured to pad the control symbol to one or more of the received one or more Infiniband signal streams, wherein the combined data rate of the received one or more Infiniband signal streams after being padded with the control symbol is substantially equal to the data rate of the serialized multiplexed one or more parallel data streams.
In another aspect, the time division multiplexed one or more parallel data streams may include two or more output signal streams, wherein the two or more output signal streams each have a data rate lower than the data rate of the received one or more Infiniband signal streams.
In yet another aspect, when the time division multiplexed one or more parallel data streams are comprised entirely of control symbols, the received one or more Infiniband signal streams is an idle stream.
The input Infiniband network communication interface may include one or more of a single data rate Infiniband port, a double data rate Infiniband port, or a quad data rate Infiniband port, and the output Infiniband network communication interface may include one or more of a single data rate Infiniband port, a double data rate Infiniband port, or a quad data rate Infiniband port.
In another embodiment, an apparatus may comprise, a means for receiving one or more Infiniband signal streams, a means for de-serializing the received one or more Infiniband signal streams to generate a corresponding one or more parallel data streams, means for de-serializing further including means for selectively adding or deleting a control symbol associated with a quiescent period of the one or more received signal streams, a means for time-division multiplexing the generated one or more parallel data streams, and a means for serializing the time division multiplexed one or more parallel data streams.
Various other modifications and alterations in the structure and method of operation of this disclosure will be apparent to those skilled in the art without departing from the scope and spirit of the disclosure. Although the disclosure has been described in connection with specific preferred embodiments, it should be understood that the disclosure as claimed should not be unduly limited to such specific embodiments. It is intended that the following claims define the scope of the present disclosure and that structures and methods within the scope of these claims and their equivalents be covered thereby.
Number | Name | Date | Kind |
---|---|---|---|
5765032 | Valizadeh | Jun 1998 | A |
5835494 | Hughes et al. | Nov 1998 | A |
5883893 | Rumer et al. | Mar 1999 | A |
5959968 | Chin et al. | Sep 1999 | A |
5991817 | Rowett et al. | Nov 1999 | A |
6061349 | Coile et al. | May 2000 | A |
6104717 | Coile et al. | Aug 2000 | A |
6163543 | Chin et al. | Dec 2000 | A |
6298061 | Chin et al. | Oct 2001 | B1 |
6430155 | Davie et al. | Aug 2002 | B1 |
6430201 | Azizoglu et al. | Aug 2002 | B1 |
6512744 | Hughes et al. | Jan 2003 | B1 |
6643267 | Karia et al. | Nov 2003 | B1 |
6665497 | Hamilton-Gahart et al. | Dec 2003 | B1 |
6675200 | Cheriton et al. | Jan 2004 | B1 |
6757348 | Vila et al. | Jun 2004 | B1 |
6826196 | Lawrence | Nov 2004 | B1 |
6845102 | Bendelac et al. | Jan 2005 | B1 |
6993048 | Ah Sue | Jan 2006 | B1 |
6996108 | Budhraja | Feb 2006 | B1 |
7110363 | Lawrence et al. | Sep 2006 | B1 |
7111102 | Doak et al. | Sep 2006 | B2 |
7114005 | McDaniel | Sep 2006 | B2 |
7133416 | Chamdani et al. | Nov 2006 | B1 |
7231518 | Bakke | Jun 2007 | B1 |
7254647 | Mann | Aug 2007 | B2 |
7295572 | Haapala | Nov 2007 | B1 |
7310695 | Epps et al. | Dec 2007 | B2 |
7346058 | Natarajan et al. | Mar 2008 | B1 |
7366902 | Bakke | Apr 2008 | B2 |
7382805 | Raza et al. | Jun 2008 | B1 |
7433948 | Edsall et al. | Oct 2008 | B2 |
7433988 | Doak et al. | Oct 2008 | B2 |
7437507 | Sharma et al. | Oct 2008 | B2 |
7656905 | Sheth et al. | Feb 2010 | B2 |
7995696 | Norrie | Aug 2011 | B1 |
20030099260 | Bunton | May 2003 | A1 |
20030149783 | McDaniel | Aug 2003 | A1 |
20030235215 | Carrel et al. | Dec 2003 | A1 |
20040156390 | Prasad et al. | Aug 2004 | A1 |
20050025075 | Dutt et al. | Feb 2005 | A1 |
20050213581 | Ah Sue | Sep 2005 | A1 |
20060087989 | Gai et al. | Apr 2006 | A1 |
20060098681 | Cafiero et al. | May 2006 | A1 |
20060101140 | Gai et al. | May 2006 | A1 |
20060155831 | Chandrasekaran | Jul 2006 | A1 |
20060168192 | Sharma et al. | Jul 2006 | A1 |
20060251067 | DeSanti et al. | Nov 2006 | A1 |
20060268943 | DeCusatis et al. | Nov 2006 | A1 |
20060277361 | Sharma et al. | Dec 2006 | A1 |
20060294234 | Bakke et al. | Dec 2006 | A1 |
20070094464 | Sharma et al. | Apr 2007 | A1 |
20070094465 | Sharma et al. | Apr 2007 | A1 |
20070094466 | Sharma et al. | Apr 2007 | A1 |
20070101134 | Parlan et al. | May 2007 | A1 |
20070143552 | Rastogi et al. | Jun 2007 | A1 |
20080273880 | Wang et al. | Nov 2008 | A1 |
20090245110 | Connolly et al. | Oct 2009 | A1 |
20090257514 | Connolly et al. | Oct 2009 | A1 |
Number | Date | Country | |
---|---|---|---|
20100103954 A1 | Apr 2010 | US |