Embodiments of the invention generally relate to the field of electronic systems, and more particularly, to a method and apparatus for signaling virtual channel support in communication networks.
The invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements and in which:
a is a graphical illustration of a BVC flow control credit initialization data link layer packet (DLLP) format, according to one embodiment of the invention;
b is a graphical illustration of an OVC flow control credit initialization DLLP format, according to one embodiment of the invention;
c is a graphical illustration of a MVC flow control credit initialization DLLP format, according to one embodiment of the invention;
a is a graphical illustration of a relationship between VC Index and VC ID, according to one embodiment of the invention;
b is a graphical illustration of a exchanging flow control credit initialization DLLPs between two nodes, according to one embodiment of the invention; and
Embodiments of the invention are generally directed to a method and apparatus for signaling virtual channel support in communication networks. In accordance with one example embodiment, a virtual channel (VC) support manager is introduced herein. As described more fully below, the innovative VC support manager is operable to signal support for one or more virtual channels of a given type on a point-to-point communication link with another node, based on the content of a data packet received from another node on the point-to-point communication link.
Data and/or instructions (hereinafter referred to as “data”) may be transmitted through point-to-point communication link 101 in electronic system 100 from a source node to a destination node. For example, the source node may be endpoint 112 and the destination node may be another endpoint 112. Data may also be transmitted on point-to-point communication link 101 from the source node to the destination node through an intermediary node or a series of intermediary nodes, such as switch element 114. In that regard, data may also be transmitted on more that one point-to-point communication link 101 between an endpoint 112 through a switch element 114 to another endpoint 112 (as shown in
According to an example embodiment, virtual channels are utilized to facilitate the efficient transmission of data on point-to-point communication link 101. These virtual channels provide a means of supporting multiple independent logical communication channels on point-to-point communication link 101. Thus, for example, data can be logically channeled by multiplexing data streams onto point-to-point communication link 101 between endpoint 112 and switch element 114.
Before data can be transmitted on point-to-point communication link 101, adequate queue resources are needed by endpoint 112 and switch element 114 to support a given virtual channel of a given type on the link. As will be explained in more detail below, an indication of adequate queue resources (i.e. sufficient available buffer capacity to handle data on a given virtual channel) is indicated or communicated to endpoint 112 and switch element 114 by exchanging data packets on point-to-point communication link 101.
Various virtual channel types assist in the efficient transmission of data on point-to-point communication link 101, such as bypass capable virtual channels (BVC), ordered-only virtual channels (OVC), or multicast virtual channels (MVC). In an example embodiment, a BVC is supported by two types of queue resources, a bypass queue and an ordered queue. An ordered queue is a first-in-first out (FIFO) queue. A bypass queue is a separate FIFO queue in which data that is marked as “bypassable” is placed. By placing the “bypassable” data in the bypass queue, and placing other data in the ordered queue, the other data can continue to pass through the ordered queue should the “bypassable” packets become stalled or delayed, thus avoiding potential data deadlocks on point-to-point communication link 101. An OVC is supported by one type of queue resource, namely, a single ordered queue.
BVC and OVC virtual channel types are used to transmit unicast data (data addressed to one destination). MVC virtual channel types are used to transmit multicast data (data addressed to one or more destinations). MVC virtual channel types use one type of queue resource, a single FIFO queue, which is similar to the resources needed to support an OVC. However, since multicast data, as opposed to unicast data, is routed through MVC virtual channel types on point-to-point communication link 101, the single FIFO queue is deemed a multicast queue.
In an example embodiment, communication protocols are introduced which, as will be developed more fully below, support one or more innovative features including, but not limited to, indicating support for one or more virtual channels of a given type on point-to-point communication link 101. This communication protocol may be used by VC support manager(s) 116 to determine a switch element 114 and/or an endpoint 112's support for one or more virtual channels on point-to-point communication link 101.
In
As used herein, control logic 220 controls the overall operation of VC support manager 200 and is intended to represent any of a wide variety of logic device(s) and/or executable content to implement the control of VC support manager 200, described herein. In this regard, control logic 220 may well be comprised of a microprocessor, network processor, microcontroller, FPGA, ASIC, or executable content to implement such control features, and/or any combination thereof. In alternate embodiments, the features and functionality of control logic 220 may well be implemented within signal engine 210.
Control logic 220 selectively invokes an instance of signal engine 210 to signal whether each node or a node supports one or more virtual channels of a given type on a point-to-point communication link between two nodes based, at least in part, on the content of DLLPs exchanged between the nodes.
As used herein, memory 230 is intended to represent a wide variety of memory media including, but not limited to volatile memory, non-volatile memory, flash and programmatic variables or states. According to an example embodiment, memory 230 is used by signal engine 210 to temporarily store information related to one or more node's resources to support one or more virtual channels of a given type on a point-to-point communication link. In this regard, memory 230 includes virtual channel queue resources tables with one or more entries for placing information related to the node's or both nodes queue resources to support one or more virtual channels of a given type on the point-to-point communication link.
Memory 230 may also include memory registers to store bit flags which are asserted or de-asserted by signal engine 210 to signal support for one or more virtual channels of a given type on the point-to-point communication link between the nodes.
Memory 230 may also store executable content. The executable content is used by control logic 220 to implement an instance of signal engine 210 to exchange DLLPs between the nodes and signal a node's virtual channel support, based on the content of the exchanged DLLPs.
As used herein, I/O interfaces 240 provides a communications interface between VC support manager 200 and an electronic system. For example, VC support manager 200 is implemented as an element of a computer system, wherein I/O interfaces 240 provides a communications interface between VC support manager 200 and the computer system via a communication channel. In this regard, control logic 220 can receive a series of instructions from application software external to VC support manager 200 via I/O interfaces 240. In that regard, the series of instructions may invoke control logic 220 to implement one or more features of signal engine 210.
In an example embodiment, VC support manager 200 includes one or more application(s) 250 to provide internal instructions to control logic 220. As used herein, such application(s) 250 may well be invoked to generate a user interface, e.g., a graphical user interface (GUI), to enable administrative features, and the like. In alternate embodiments, one or more features of signal engine 210 may well be implemented as an application(s) 250, selectively invoked by control logic 220 to invoke such features.
In one embodiment, a VC support manager 200 is located within switch element 114. In that regard, VC support manager 200 invokes an instance of read feature 212 to read a DLLP received at switch element 114 over point-to-point communication link 101, for example, from endpoint 112. Additionally, VC support manager 200 also selectively invokes an instance of resource feature 216 to determine the queue resources switch element 114 supports for a virtual channel of a given type on point-to-point communication link 101. In an example implementation, as will be explained in more detail below, the invoking of read feature 212 occurs concurrently with the invoking of resource feature 216.
Resource feature 216 populates a first temporary virtual channel queue resources table, e.g. maintained in memory 230, with one or more entries indicating the amount of queue resources switch element 114 supports for the virtual channel of a given type on point-to-point communication link 101. As introduced above and explained in more detail below, such queue resources indicate support for one or more virtual channels of a given type that includes BVC, OVC and/or MVC virtual channel types.
Once resource feature 216 populates a first temporary virtual channel queue resources table for switch element 114, VC support manager 200 then selectively invokes an instance of transmit feature 214. Transmit feature 214, based, at least in part, on the queue resources populated in the first temporary virtual channel queue resources table, generates one or more DLLPs and transmits the DLLPs to indicate switch element 114's support or lack of support for the virtual channel of a given type on point-to-point communication link 101.
Further, read feature 212, based, at least in part, on the content of the one or more DLLPs received at switch element 114, populates a second temporary virtual channel queue resources table, e.g. maintained in memory 230, with the amount of queue resources indicated in the one or more DLLPs received from a node at the other end of the point-to-point link 101, for example, endpoint 112. Once the first and second temporary virtual channel queue resources tables are populated, resource feature 216 then reads the first and second temporary virtual channel queue resources tables. Resource feature 216 then, based, at least in part, on the contents of temporary virtual channel queue resources tables, signal whether the virtual channel of a given type on point-to-point communication link 101 is supported.
This process of exchanging DLLPs continues until switch element 114's support is determined for each of the one or more virtual channels on point-to-point communication link 101.
In one embodiment, VC support manager 200 may be located within endpoint 112. Thus, VC support manager 200 determines endpoint 112's support for one or more virtual channels of a given type on point-to-point communication link 101. In other embodiments, VC support manager 200 may be located outside of endpoint 112 and switch element 114 to determine and signal both nodes' support for one or more virtual channels of a given type on point-to-point communication link 101.
According to an example embodiment, resource feature 216 signals support of a virtual channel of a given type by asserting or de-asserting a bit flag in a memory register stored in a memory (i.e. in memory 230) accessible to elements of VC support manager 200 or elements external to VC support manager 200 via I/O interfaces 240.
a is a graphical illustration of a BVC flow control credit initialization data link layer packet (DLLP) format, according to one embodiment of the invention. InitFC-BVC 310 is depicted comprising a 32-bit BVC initial credit DLLP format, although the invention is not limited to a 32-bit format.
In an example embodiment, one or more InitFC-BVC 310 DLLPs are transmitted by endpoint 112 or switch element 114 to indicate support on a virtual channel of BVCs on point-to-point communication link 101. The InitFC-BVC 310 DLLPs each contain two fields to indicate BVC support: a Bypass Queue Credits field and an Ordered Queue Credits field. As will be explained in more detail in
As mentioned previously, in order to support a BVC, both endpoint 112 and switch element 114 need adequate queue resources to support both a bypass and an ordered queue on the virtual channel. Serving as an advertisement of bypass and ordered queue depths or capacities, a non-zero value within both the Bypass Queue and Ordered Queue Credits fields of each InitFC-BVC 310 DLLP indicates a BVC is supported for the given virtual channel identifier on point-to-point communication link 101.
b is a graphical illustration of an OVC flow control credit initialization DLLP format, according to one embodiment of the invention. In
In an example embodiment, one or more InitFC-OVC 320 DLLPs are transmitted by endpoint 112 or switch element 114 to indicate support of OVCs on one or more virtual channels on point-to-point communication link 101. The InitFC-OVC 320 DLLPs each contain two fields to indicate OVC support for two virtual channels at a time. As will be explained in more detail in
Serving as an advertisement of ordered queue depth or capacity, a non-zero value within the Ordered Queue Credit field of each InitFC-OVC 320 DLLP indicates an OVC is supported for the given virtual channel identifier on point-to-point communication link 101.
c is a graphical illustration of an example MVC flow control credit initialization DLLP format, according to one embodiment of the invention. In
In an example embodiment, one or more InitFC-MVC 330 DLLPs are transmitted by endpoint 112 or switch element 114 to indicate support of MVCs on one or more virtual channels on point-to-point communication link 101. The InitFC-MVC 330 each contain two fields to indicate MVC support for two virtual channels at a time. As will be explained in more detail in
Serving as an advertisement of multicast queue depth or capacity, a non-zero value within the Multicast Queue Credit field of each InitFC-MVC 330 DLLP indicates a MVC is supported for the given virtual channel identifier on point-to-point communication link 101.
a is a graphical illustration of a relationship between VC Index and VC ID, according to one embodiment of the invention. Table 405 lists assignments for a given virtual channel identifier number (VC ID) to a virtual channel index (VC index) and further lists assignments for a range of VC index numbers to either a BVC, OVC or MVC configuration, although the invention is not limited in this regard.
b is a graphical illustration of exchanging flow control credit initialization DLLPs between two nodes, according to one embodiment of the invention. Nodes 410 and 420 are shown in
According to an example embodiment, node 410 transmits to node 420 flow control initialization DLLPs with a non-zero credit value in the appropriate queue credit fields, at least one DLLP being associated with one or more supported virtual channels. When following the VC ID assignments listed in table 405, Node 410 transmits to Node 420 5 DLLPs with non-zero credit values for VC IDs 0, 1, 8, 9, 10 and 16 (since VC IDs 8 & 9 will be transmitted by the same DLLP with VC Index 8, see table 405). Node 420 will also transmit to node 410 up to 5 DLLPs with a non-zero credit value for VC IDs 0, 1, 2, 3 and 16.
Thus, in this example embodiment, based on the content of the exchanged flow control initialization DLLPs, only VC IDs 0, 1 and 16 are commonly supported by nodes 410 and 420. Consequently, VC IDs 0, 1, and 16 are supported on the point-to-point communication link between nodes 410 and 420.
Resource feature 216, in an example embodiment, then populates a first temporary virtual channel queue resources table, i.e., located in memory 230, with one or more entries reflecting the amount of queue resources for a virtual channel of a given type that switch element 114 supports for VC(x) on point-to-point communication link 101.
Once the first virtual channel queue resources table is populated by resource feature 216, the process moves to block 520. In block 520, signal engine 210 invokes an instance of transmit feature 214. Transmit feature 214 accesses the first temporary virtual channel queue resources table and generates a DLLP for VC(x) in the format of InitFC-BVC 310, InitFC-OVC 320 or InitFC-MVC 330 based, at least in part, on the contents in the first temporary virtual channel queue resources table.
Transmit feature 214 then transmits the DLLP for VC(x) over point-to-point communication link 101. This DLLP indicates switch element 114's available queue resources for a virtual channel of a given type for VC(x) on point-to-point communication link 101.
As mentioned previously, the reading of DLLPs occurs concurrently with the transmitting of DLLPs. In this regard, the process is further explained in block 530, wherein in response to control logic 220, signal engine 210 invokes an instance of read feature 212.
In an example embodiment, a DLLP for VC(x) formatted according to InitFC-BVC 310, InitFC-OVC 320 or InitFC-MVC 330, is sent by endpoint 112 to switch element 114 on point-to-point communication link 101.
Read feature 212 reads the virtual channel identifier field, (i.e. VC Index 0-7 for BVC, VC Index 8-11 for OVC and VC Index 12-13 for MVC), and the ordered queue credits fields and/or bypass queue credits field or multicast queue credits fields of the DLLP. Once the applicable fields of the DLLP are read by read feature 212, the process moves to block 540. In block 540, read feature 212, based on the content of the fields, populates a second temporary virtual channel queue resources table (i.e. in memory 230) with the amount of queue resources for VC(x) indicated in the DLLP.
Once the first and the second temporary virtual channel queue resources tables are populated by read feature 212, as described in blocks 510 and 540, the process moves to block 550. In block 550, resource feature 216 accesses the first and second temporary virtual channel queue resources tables and based, at least in part, on the contents of the temporary virtual channel queue resources tables, determines whether VC(x) is supported (i.e. adequate queue resource credits are present to support an ordered and/or bypass or multicast queues for VC(x) on point-to-point communication link 101).
If resource feature 216 determines that VC(x) is commonly supported, the process moves to block 560. In block 560, resource feature 216 signals, by the assertion of a bit flag, that VC(x) is commonly supported by endpoint 112 and switch element 114 on point-to-point communication link 101. The bit flag is asserted in a memory register stored in a memory (i.e. memory 230). The process then starts over to determine if additional VC(x)s are supported by endpoint 112 and switch element 114 on point-to-point communication link 101.
If resource feature 216 determines that VC(x) is not supported by endpoint 112, the process moves to block 570. In block 570, resource feature 216, signals by the de-assertion of a bit flag, that VC(x) is not supported by endpoint 112 and switch element 114 on point-to-point communication link 101. The bit flag is de-asserted in a memory register stored in a memory (i.e. in memory 230). The process then starts over to determine if additional VC(x)s are supported by endpoint 112 and switch element 114 on point-to-point communication link 101.
Referring again to the block diagram of
In accordance with one embodiment, system control logic 104 controls the overall operation of electronic system 100 and is intended to represent any of a wide variety of logic device(s) and/or executable content to implement the operation of electronic system 100, described herein. In this regard, system control logic 104 may well be comprised of a microprocessor, network processor, microcontroller, FPGA, ASIC, executable content to implement such control features and/or any combination thereof.
Electronic system 100 further includes system memory 106 to store information/features offered by electronic system 100. In this regard, system memory 106 is used to store temporary variables or other intermediate information during execution of instructions by system control logic 104. As used herein, system memory 106 may well include a wide variety of memory media including but not limited to volatile memory, non-volatile memory, flash, programmable variables or states, random access memory (RAM), read-only memory (ROM), flash, or other static or dynamic storage media.
In accordance with one example embodiment, machine-readable instructions can be provided to system memory 106 from a form of machine-accessible medium. As used herein, a machine-accessible medium is intended to represent any mechanism that provides (i.e., stores and/or transmits) information in a form readable by a machine (e.g., electronic system 100). For example, a machine-accessible medium may well include ROM; RAM; magnetic disk storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals); and the like. Instructions may also be provided to system memory 106 via a remote connection through System I/O interfaces 108 (e.g., over a communication network).
Endpoint 112 represents an element of electronic system 100. Endpoint 112 may be either a source node or a destination node for data transmitted within and/or remote to electronic system 100. Endpoint 112 may well comprise one or more of a router, network processor, embedded logic, input/output port for a switch fabric and the like.
As used, herein, switch element 114 represents an element of electronic system 100 which acts as an intermediary node for data transmitted from one or more nodes located within and/or remote to electronic system 100. As used herein, switch element 114 is intended to represent any of a number of hardware and/or software element(s) to receive and transmit data. In this regard, according to one example embodiment, switch element 114 may well comprise one or more of an intermediary switch for a switch fabric, a bridge, a microprocessor, software application, embedded logic, or the like.
VC support manager(s) 116 may be encompassed within endpoint 112 and/or switch element 114. Alternatively, VC support manager(s) 116 may well be communicatively coupled to endpoint 112 and/or switch element 114 through e.g. communication channel 102 or through System I/O interfaces 108.
System I/O interfaces 108 may also enable one or more element(s), e.g., system control logic 104, to interact with input and/or output devices, for example, a mouse, keyboard, touchpad, cathode ray tube monitor, liquid crystal display, etc.
According to one example embodiment, VC support manager(s) 116 assistance in signaling a node's (i.e. endpoint 112 and/or switch element 114) support of one or more virtual channels of a given type on point-to-point communication link to another node may well be implemented in hardware, software, firmware, or any combination thereof. In this regard, VC support manager(s) 116 may well be implemented as one or more of an ASIC, special function controller or processor, FPGA, other hardware device and firmware or software to perform at least the functions described herein.
In the previous descriptions, for the purpose of explanation, numerous specific details were set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art, that the invention can be practiced without these specific details. In other instances, structures and devices were shown in block diagram form in order to avoid obscuring the invention.
References made in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. Thus, the appearances of the phrase “in one embodiment” appearing in various places throughout the specification are not necessarily all referring to the same embodiment. Likewise, the appearances of the phrase “in another embodiment,” or “in an alternate embodiment” appearing in various places throughout the specification are not all necessarily referring to the same embodiment.
While the invention has been described in terms of several embodiments, those of ordinary skill in the art will recognize that the invention is not limited to the embodiments described, but can be practiced with modification and alteration within the spirit and scope of the appended claims. The description is thus to be regarded as illustrative of, rather than limiting the scope and coverage of the claims appended hereto.
The present application is a continuation of U.S. patent application Ser. No. 10/888,212, filed Jul. 9, 2004, entitled “Method and apparatus for signaling virtual channel support in communication networks”, issued as U.S. Pat. No. 8,098,669 on Jan. 17, 2012, which claims priority to and benefit of U.S. Provisional Application No. 60/492,566, filed Aug. 4, 2003, which are incorporated herein by reference in their entirety and for all purposes.
Number | Name | Date | Kind |
---|---|---|---|
4975905 | Mann et al. | Dec 1990 | A |
5353282 | Dormer et al. | Oct 1994 | A |
5574934 | Mirashrafi et al. | Nov 1996 | A |
5625779 | Solomon et al. | Apr 1997 | A |
5740385 | Hayek et al. | Apr 1998 | A |
5742603 | Shafir et al. | Apr 1998 | A |
5745837 | Fuhrmann | Apr 1998 | A |
5777984 | Gun et al. | Jul 1998 | A |
5953338 | Ma et al. | Sep 1999 | A |
6081848 | Grun et al. | Jun 2000 | A |
6212589 | Hayek et al. | Apr 2001 | B1 |
6266345 | Huang | Jul 2001 | B1 |
6285659 | Feuerstraeter et al. | Sep 2001 | B1 |
6304549 | Srinivasan et al. | Oct 2001 | B1 |
6317803 | Rasmussen et al. | Nov 2001 | B1 |
6393506 | Kenny | May 2002 | B1 |
6442632 | Hayek et al. | Aug 2002 | B1 |
6512767 | Takeda et al. | Jan 2003 | B1 |
6647474 | Vishlitzky et al. | Nov 2003 | B2 |
6691192 | Ajanovic et al. | Feb 2004 | B2 |
6999421 | Holzworth et al. | Feb 2006 | B1 |
7058008 | Wilson et al. | Jun 2006 | B1 |
7089234 | Dugan et al. | Aug 2006 | B2 |
7107335 | Arcieri et al. | Sep 2006 | B1 |
7240123 | Kamboh et al. | Jul 2007 | B2 |
7376828 | Voit et al. | May 2008 | B1 |
20010056459 | Kurose et al. | Dec 2001 | A1 |
20020131412 | Shah et al. | Sep 2002 | A1 |
20020184358 | Traversat et al. | Dec 2002 | A1 |
20030076831 | Van Doren et al. | Apr 2003 | A1 |
20030115391 | Ajanovic et al. | Jun 2003 | A1 |
20030151621 | Mcevilly et al. | Aug 2003 | A1 |
20040170178 | Motobayashi | Sep 2004 | A1 |
Number | Date | Country |
---|---|---|
1102437 | May 2001 | EP |
0203622 | Jan 2002 | WO |
03019391 | Mar 2003 | WO |
2005015830 | Feb 2005 | WO |
Entry |
---|
International Search Report and Written opinion received for PCT Application No. PCT/US2004/025213, mailed on Jun. 12, 2004, 17 pages. |
International Preliminary Report on Patentability and Written opinion received for PCT Application No. PCT/US2004/025213, mailed on Feb. 16, 2006, 12 pages. |
PCI Express™ Base Specification, Revision 1.0a, Apr. 15, 2003, 426 pages. |
InfiniBand™ Architecture Specification, vol. 1, Release 1.1, Final Release, Nov. 6, 2002, 1727 Pages. |
InfiniBand™ Architecture Specification, vol. 2, Release 1.1, Final, Nov. 6, 2002, 700 pages. |
Number | Date | Country | |
---|---|---|---|
20120113988 A1 | May 2012 | US |
Number | Date | Country | |
---|---|---|---|
60492566 | Aug 2003 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 10888212 | Jul 2004 | US |
Child | 13352291 | US |