This application claims the benefit of India Patent Application No. 202111040728 filed Sep. 8, 2021, the disclosure of which is incorporated herein by reference in its entirety.
This invention relates generally to switches, and more specifically to graphics processing unit (GPU) switching systems.
Wang, Wen-Fong & Li, Jhih-Sian & Lu, Ching-Sung. (2011). Design of a Crosspoint Queued router for a micro-network in SoCs. 10.1109/ICACC.2011.6016470, whose abstract is available online, describes that “Due to the advancement of manufacturing process for integrated circuits, more functional units can be integrated into a System on Chip (SoC). It causes the challenge of communication aspect in a SoC. Therefore, the design of Network on Chip (NoC) was proposed. Most research of NoC employs a kind of Input-Queued (IQ) switch architecture with VOQ (Virtual Output Queue) scheme.”
A virtual channel router is described in Kavaldjiev, N. & Smit, G. J. M. & Jansen, P. G. (2004). A virtual channel router for on-chip networks. 289-293. 10.1109/SOCC.2004.1362438, whose abstract is available online.
Example methods for virtual channel flow control are described in W. Dally, Virtual Channel Flow Control, IEEE Trans. Parallel Distributed Syst., vol. 3, no. 2, pp. 194-205, March 1992.
Crossbar switches are described at the following https online location: en.wikipedia.org/wiki/Crossbar switch.
A iSLIP Scheduling Algorithm is described in N. McKeown, The ISLIP Scheduling Algorithm for Input-Queued Switches, IEEE Transactions on Networking, Vol 7, pp. 188-201, April 1999.
This list of Changes and New Feature History of Nvidia products: https://docs.mellanox.com/m/view-rendered-page.action?abstractPageId=43718262 refers to support for configuring Virtual Output Queuing (VoQ).
This research paper, available at the following https online location:
research.nvidia.com/sites/default/files/pubs/2018-11_Exploiting-Idle-Resources/Stashing_SC18.pdf describes 5×5 tile crossbars which use a virtual-output-queued (VOQ) architecture, with four packets worth of storage per virtual channel.
The research paper by Wladek Olesinski jointly with Hans Eberle of NVIDIA, available at the following https www location: researchgate.net/publication/221171914_Scalable_Alternatives_to_Virtual_Output_Queuing describes that virtual output queues (VOQs) are commonly used to avoid head of line blocking in switches. Similarly, Wikipedia's entry on “Virtual output queueing” describes that “Virtual output queueing (VOQ) is a technique used in certain network switch architectures where, rather than keeping all traffic in a single queue, separate queues are maintained for each possible output location. It addresses a common problem known as head-of-line blocking. In VOQ, the physical buffer of each input port maintains a separate virtual queue for each output port. Therefore congestion on an egress port will block only the virtual queue for this particular egress port. Other packets in the same physical buffer destined to different (non-congested) output ports are in separate virtual queues and can therefore still be processed. In a traditional setup, the blocked packet for the congested egress port would have blocked the whole physical buffer, resulting in head-of-line blocking . . . . VOQ can achieve 100% throughput performance with an effective scheduling algorithm. This scheduling algorithm should be able to provide a high speed mapping of packets from inputs to outputs on a cycle-to-cycle basis. The VOQ mechanism provides throughput at a much higher rate than the crossbar switches without it”.
Virtual channels are described e.g. in this online https reference: ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=134508.
The reference at the following http online location: cva.stanford.edu/classes/ee382c/ee482b/research/jhsu.pdf
describes scheduling in which “each input is routed to every output that is requested in the buffer. The randomness in choosing the grant in step 2 ensures that . . . no input-output flow is starved.”
Data buffers are described generally in Wikipedia's entry on “Data buffer”. The terms fabric and matrix are used in the art (e.g. in the following https www location: sdxcentral.com/networking/virtualization/definitions/what-is-networking-switch-fabric/) to refer to the topology of a switch (e.g. the IQ switch described herein) which determines which source is routed to which destination.
Scheduling algorithms are useful e.g. in crossbar switches.
Certain embodiments seek to provide practical high-performance systems, by providing a scheduling algorithm with all or any appropriate subset of the following properties.
High Throughput: An algorithm that keeps the backlog low in the Virtual Output Queues aka VOQ's; ideally, the algorithm will sustain an offered load up to 100% on each input and output; and/or
Starvation Free: The algorithm typically does not allow a non-empty Virtual Output Queue aka VOQ to remain unserved indefinitely; and/or
Fast: To achieve the highest bandwidth switch, it is desired that the scheduling algorithm does not become the performance bottleneck; the algorithm therefore typically finds a match as quickly as possible; and/or
Convenient to Implement: algorithm which is fast in practice, and may be implemented in special-purpose hardware, preferably within a single chip.
Certain embodiments seek to provide virtual channel RR-LRU iSLIP arbitration, e.g. for “credit at dispatch” switches, which is starvation free. Arbiter apparatus uses an RR-LRU arbitration scheme to make connections, over which to transmit at least one packet through a switch, between input and output ports, and in that RR-LRU arbitration scheme, a first set of arbiters uses a Round Robin (RR) arbitration scheme and a second set of arbiters uses a Least Recently Used (LRU) scheme.
Certain embodiments seek to provide plural virtual channel specific round robin arbiters such that when credit of any virtual channel runs out, a credit-at-dispatch scheme may mask all requests to that virtual channel's specific round robin arbiter whose round robin pointer may be stuck (frozen) until credit is returned to that virtual channel, thereby to ensure that no VC requests remain unserved indefinitely, thanks to credit-at-dispatch request masking.
Certain embodiments seek to provide arbiter apparatus serving an input queued switch typically having plural input ports and/or plural output ports and/or plural virtual channels and/or credit-based flow control, which typically ensures that at least one, or typically every one of the plural input ports, transmits at most 1 cell at a time, and/or ensures that at least one, or typically every one of, the plural output ports, receives at most 1 cell at a time and/or ensures that at least one (typically each) output port, from among the plural output ports, receives a cell over only 1 virtual channel (VC) (from among the plural virtual channels) at a time, thereby to function as a dispatch unit.
At least the following embodiments are included in the scope of the present invention:
Embodiment 1. A switching system having plural input ports and plural output ports and comprising: an input queued switch with plural virtual channels, only one of which can, at a given time, access a given output port from among the plural output ports, wherein the input queued switch includes arbiter apparatus which controls the plural input ports and plural output ports to ensure that at least one input port, from among the plural input ports, transmits at most one cell at a time, and also that at least one output port, from among the plural output ports, which receives a cell, receives that cell over only 1 virtual channel (VC) from among the plural virtual channels, and wherein the arbiter apparatus functions as a dispatch unit in which at least one output port, from among the plural output ports, receives at most one cell at a time.
Embodiment 2. A system according to any preceding embodiment wherein the arbiter apparatus comprises:
a first set of arbiters which selects, for at least one output port O,
at least one input port from among the plural input ports; and
at least one virtual channel from among the plural virtual channels characterized in that the at least one input port has at least one queued cell, whose destination is output port O, in each of the at least one virtual channel, and
a second set of arbiters, which selects, for each input port I from among the plural input ports, at least one output port O, from among the output ports, characterized in that the first set of arbiters selected input port I for output port O.
Embodiment 3. A system according to any preceding embodiment wherein the first set includes, for each of plural output ports, a subset of arbiters to select, at least once, an input port I from among input ports which is requesting that output port as the input port I's destination and a virtual channel V from among virtual channels which is requesting that output port as virtual channel V's destination, thereby to provide plural subsets of arbiters.
Embodiment 4. A system according to any preceding embodiment wherein at least one subset of arbiters includes an arbiter per virtual channel, thereby to define a DV arbiter per virtual channel, thus providing plural DV arbiters.
Embodiment 5. A system according to any preceding embodiment wherein the at least one subset of arbiters also includes virtual channel select logic to select among the plural arbiters.
Embodiment 6. A system according to any preceding embodiment wherein plural input port requests all to a single virtual channel and whose destinations all comprise a single output port, are all connected to a single DV arbiter in a single subset of arbiters, whereas input port requests to different virtual channels, but whose destinations all comprise a single output port, are connected to different DV arbiters in a single subset of arbiters.
Embodiment 7. A system according to any preceding embodiment wherein at least one DV arbiter uses an arbitration scheme to select among input ports, and wherein the arbitration scheme used by the DV arbiter comprises a Round-robin scheme which uses circular priority to grant the requests.
Embodiment 8. A system according to any preceding embodiment wherein the DV arbiter updates the circular priority only after receiving an accept, from the second set of arbiters, through the virtual channel select logic.
Embodiment 9. A system according to any preceding embodiment wherein the virtual channel select logic uses an arbitration scheme to select a DV arbiter to pass a granted input port, from among the plural input ports, to the second set of arbiters, from among plural DV arbiters which each have a granted source.
Embodiment 10. A system according to any preceding embodiment wherein the arbitration scheme used by the virtual channel select logic comprises a round robin arbitration scheme.
Embodiment 11. A system according to any preceding embodiment wherein the DV arbiter selected by the virtual channel select logic generates a WRAP signal when all active input ports in that DV arbiter are granted or before again granting an input port.
Embodiment 12. A system according to any preceding embodiment wherein once the VC select logic has selected a DV arbiter, thereby to define a currently selected DV arbiter, the VC select logic moves to a new DV arbiter only after receiving a WRAP signal from the currently selected DV arbiter.
Embodiment 13. A system according to any preceding embodiment wherein the virtual channel select logic passes an accept to a selected DV arbiter, when and only when the second set of arbiters accepts a grant from the first set of arbiters.
Embodiment 14. A system according to any preceding embodiment wherein at least one arbiter in the second set of arbiters uses a priority scheme in which a request, which has a given priority, receives a grant before other requests, which have priorities lower than the given priority, receive grants.
Embodiment 15. A system according to any preceding embodiment wherein the switch uses credit-based flow control, whereby packets are transmitted to an output port, from among the output ports, only when buffer space is known to exist at the destination of the output port, to avoid packet drop in at least one switch.
Embodiment 16. A system according to any preceding embodiment wherein, to support the credit-based flow control, the switch maintains a counter for every destination per virtual channel, which keeps track of buffer space per destination by decrementing the counter for every packet sent to a given destination and incrementing the counter for every credit returned from the given destination, and wherein credits are returned from a destination D from among the output ports, whenever a destination buffer at destination D is freed up.
Embodiment 17. A system according to any preceding embodiment wherein Virtual Output Queues store incoming packets, and wherein each destination's and virtual channel's credit counter generates a ready indication whenever a given destination and a given virtual channel have enough credit to accommodate a packet, and wherein, when the ready indication is asserted, requests to transmit packets through the switch in all input ports' Virtual Output Queue of the destination and virtual channel are exposed to the dispatch unit, else all requests to transmit packets through the switch are masked, thereby to provide a “credit at dispatch” scheme of credit-based flow control.
Embodiment 18. A switching method comprising:
providing an input queued switch with plural virtual channels, only one of which can, at a given time, access a given output port from among plural output ports; and
using an arbiter apparatus to control plural input ports and the plural output ports, to ensure that at least one input port, from among the plural input ports, transmits at most one cell at a time, and also that at least one output port, from among the plural output ports, which receives a cell, receives that cell over only one virtual channel (VC) from among the plural virtual channels,
thereby to function as a dispatch unit in which at least one output port, from among the plural output ports, receives at most one cell at a time.
Embodiment 19. A system according to any preceding embodiment wherein the system has N inputs, M outputs, and K virtual channels and wherein the arbiter apparatus finds a set of up to min (M,N) cells to transmit over the switch.
Embodiment 20. A system according to any preceding embodiment and wherein the priority scheme comprises a round robin scheme.
Embodiment 21. A system according to any preceding embodiment and wherein the priority scheme comprises a Least Recently Used (LRU) scheme in which at least one request which got a grant less recently, has higher priority relative to requests which got grants more recently.
Embodiment 22. A system according to any preceding embodiment wherein, to support the credit-based flow control, the switch maintains a counter for every destination per virtual channel, which keeps track of buffer space per destination by incrementing the counter for every packet sent to a given destination, and decrementing the counter for every credit returned from the given destination and wherein credits are returned from a destination D from among the output ports, whenever a destination buffer at destination D is freed up.
Embodiment 23. A system according to any preceding embodiment wherein the virtual channel select logic does not pass the given virtual channel's DV arbiter's grants when the ready indication is not high.
Embodiment 24. A system according to any preceding embodiment wherein the DV arbiter does not grant inputs when the ready indication is not high.
Embodiment 25. A system according to any preceding embodiment wherein the counter is incremented each time a packet is sent to the given destination and is decremented each time a credit returns from the given destination.
Embodiment 26. A system according to any preceding embodiment wherein at least one arbiter, corresponding to a source S, in the second set of arbiters, uses a priority scheme to accept at least one grant, from among plural grants to source S.
Embodiment 27. A system according to any preceding embodiment wherein at least one arbiter in the second set of arbiters uses a priority scheme to accept a grant from among the grants of DCS which have granted to this source.
Embodiment 28. A system according to any preceding embodiment wherein the arbiter apparatus ensures that each input port, from among the plural input ports, transmits at most one cell at a time, and also that each output port, from among the plural output ports, which receives a cell, receives that cell over only 1 virtual channel (VC) from among the plural virtual channels, and wherein the arbiter apparatus functions as a dispatch unit in which each output port, from among the plural output ports, receives at most one cell at a time.
Embodiment 29. A switching method comprising, in an input queued switch with plural virtual channels, only one of which can, at a given time, access a given output port from among plural output ports:
controlling plural input ports and the plural output ports, to ensure that at least one input port, from among the plural input ports, transmits at most one cell at a time, and also that at least one output port, from among the plural output ports, which receives a cell, receives that cell over only one virtual channel (VC) from among the plural virtual channels,
whereby at least one output port, from among the plural output ports, receives at most one cell at a time.
The following terms may be construed either in accordance with any appropriate definition thereof appearing in literature in the relevant field of technology or in accordance with the specification, or to include in their respective scopes, the following:
Arbiters: include electronic devices that allocate access to shared resources. Arbiters mentioned herein may comprise synchronous arbiters. Typically, each arbiter herein is a 2D arbiter aka “2D ARB”.
Arbitration scheme: intended to include any scheme which determines the efficiency of (say) a crossbar switch. Commonly used arbitration schemes are Round Robin (RR), Least Recently Used (LRU).
Arbitration e.g. VOQ arbitration: intended to include scheduling of virtual output queues—typically including resolving plural concurrent access wishes.
Space: intended to include availability of a resource.
Area: intended to include a Si (silicon) area where a given resource may be implemented in a given design.
Credit: An available resource at a given space. Flow controls are used to prevent any packet drop within switches. In creditor-based flow control, packets are dispatched to destination if destination is guaranteed to have space or credit to accommodate the packet. Thus, credit may be due to an interface protocol used to implement a given resource.
DCS set or DCS: In arbiter apparatus, a first set of arbiters which selects at least one source and at least one virtual channel requesting at least one destination.
SCD set or SCD: a second set of arbiters which typically selects among destinations which have selected at least one source.
Destination DCS: in the first set, a subset of arbiters, for each of plural destinations, which may select, at least once, a source from among sources requesting that destination and a virtual channel from among virtual channels requesting that destination. Typically, there are plural subsets of arbiters, thus plural destination DCSs.
DV arbiter: Typically at least one subset of arbiters includes an arbiter (aka DV arbiter) per virtual channel, which typically yields plural arbiters (aka plural DV arbiters).
Credit counter: A switch typically uses credit-based flow control, in which packets are transmitted to a destination only when buffer space is known to exist at the destination. The switch typically maintains a counter (aka credit counter) to support the credit-based flow control, for every destination per virtual channel.
iSLIP: referred to herein as one example of a round-robin matching algorithm or of an iterative round-robin scheduling algorithm.
iSLIP arbitration: According to the reference at the following online http location: cva.stanford.edu/classes/ee382c/ee482b/research/jhsu.pdf:
“ISLIP steps . . . are:
1. Request. Each input sends a request to every output for which it has a queued cell.
2. Grant. If an output receives any requests, it chooses the one that appears next in a fixed, round-robin schedule starting from the highest priority element. The output notifies each input whether or not its request was granted. The pointer to the highest priority element of the round-robin schedule is incremented (modulo N) to one location beyond the granted input if and only if the grant is accepted in Step 3.
3. Accept. If an input receives a grant, it accepts the one that appears next in a fixed, round-robin schedule starting from the highest priority element. The pointer to the highest priority element of the round robin schedule is incremented (modulo) N to one location beyond the accepted output.”
Thus typically, iSLIP uses round robin arbitration.
The following online location: nms.lcs.mit.edu/6829-papers/islip-ton.pdf also describes an iSLIP algorithm.
RR-LRU iSLIP arbitration: Least Recently Used (LRU) and Round Robin (RR) arbitration schemes are known. In conventional nomenclature for iSLIP arbitration schemes, the DCS arbitration scheme precedes the SCD arbitration scheme. For example, |RR-LRU iSLIP means the DCS has an RR aka round robin arbitration scheme whereas the SCD has an LRU scheme. It is appreciated that the LRU-LRU iSLIP arbitration scheme suffers from severe performance loss hence is from that point of view at least, undesirable compared to the RR-LRU iSLIP arbitration scheme on which embodiments herein are focused.
A “request” (e.g. to transmit packets through a switch, or a virtual channel “requesting” a destination) occurs in iSLIP arbitration in which there is a “request” stage during which each input port sends a request to at least one (typically each) output port O for which that input port has at least one queued cell whose destination is O. According to certain embodiments, each virtual channel (VC) of each input port sends a request to at least one (typically each) output port O for which that virtual channel (VC) of that input port has at least one queued cell whose destination is O.
A request is “active” when, for a given destination and a given virtual channel, the request is waiting to be served or dispatched (as opposed to a request which has already been dispatched which is no longer “active”.
Cell: plural portions of a packet; cells are the granularity at which a switching system works.
Circular priority: In round robin schemes, requests are typically granted in a circular priority, starting from the request adjacent or next to or coming after the most recently granted request.
“Pre-creditor” crediting scheme: typically, Virtual Output Queues aka VOQs store incoming requests. A credit counter generates “ready” whenever a destination corresponding to that credit counter has enough credit to accommodate a request. In a typical “pre-creditor” crediting scheme, aka “pre creditor scheme”, credit-based flow control is supported by crediting incoming requests in Virtual Output Queues and placing the incoming requests in a ready queue by deducting credit from the credit counter. The dispatch unit dispatches the packets in the ready queue based on a suitable arbitration scheme e.g. as described herein. The LRU arbitration scheme may be used to credit requests. This typically increases buffer size(credit) needed at destination hence area significantly because:
Credits are consumed before dispatching and/or
Ready queue increases the structural latency and/or
Destination typically needs extra credit (buffer space) in order to credit all Virtual Output Queue requests.
“Credit at dispatch”: a scheme in which credit counter/s generates “ready” (a ready indication) whenever a given destination has enough credit to accommodate a request. only when “ready” is asserted, are requests from sources exposed to a dispatch unit. Credit counters may be per destination per virtual channel (VC) and may mask/unmask requests in corresponding virtual channels (VC's) corresponding destination Virtual Output Queue.
Dispatcher: is intended to include a scheduler comprising a scheduling algorithm running on a hardware processor, or 2D ARB as described herein. The dispatcher may, in addition to the Scheduler, include a fabric (matrix) unit e.g. as shown in
Fabric: A diagram of a fabric is available at the following https link: etherealmind.com/wp-content/uploads/2011/07/what-switch-fabric-pt2-6.jpg
GPU: graphics processing unit
Si: silicon
Performance: may include a rate at which packet transmission occurs in a given switching system.
Wrap: Typically includes an Indication from arbiter that all active requests have been granted at least once since the beginning of a current arbitration window. The term “Arbitration window” as used herein may refer to a duration of time which elapses between wrap assertions or from reset to a first wrap assertion.
The terms “scheduler”, “dispatcher”, “dispatch switch” “arbiter” (e.g. 2D arbiter) and “dispatcher unit” may be interchanged herein.
The term “IQ switch” aka input queued switch typically, e.g. as shown in
The terms input, “input port” and source may be interchanged herein.
Also, the terms output, “output port”, and destination may be interchanged herein.
The terms space, area and credit may be interchanged herein.
A switching system is now described, e.g. as shown in
Typically, the input queued switch includes arbiter apparatus which controls the input ports and output ports to ensure that each input port transmits at most one cell at a time, and also that each output port which receives a cell, receives that cell over only one of the switch's virtual channels (VCs). The arbiter apparatus aka arbiter typically functions as a dispatch unit in which each output port, from among the plural output ports, receives at most one cell at a time.
Re
The Credit flow control block of
The Input Queued (IQ) switch architecture, e.g. as shown in
It is appreciated that the input queued switch e.g. of
In Input Queued architecture, buffers are deployed at the input of the switches. When a single queue is used at the input, IQ switches suffer from a problem, termed Head of Line (HOL) blocking, in which a request to a free destination is blocked by request to a destination that is busy. HOL blocking is eliminated by using Virtual Output Queuing (VOQ), where requests to each destination are stored in a separate queue. To improve network throughput and avoid deadlock, virtual channels (VCs) may be introduced. A virtual channel transports the independent packet or cell between nodes by (optionally increasing and then) dividing the resource (buffer storage) associated with each physical channel into several groups, where dividing may be done by the designer of the switch. Each virtual channel (VC) has its own set of VOQs for each output at each input. Virtual channels (VCs) may compete against one another to get access to physical channels.
Typically, a switching system is responsible for distributing resources. When distributing, typically, virtual channel resources must not be oversubscribed and/or conflict over a physical channel must be avoided.
Each input can receive up to one packet at a time which contain tag with destination details and virtual channel (VC) details. Based on virtual channel (VC) and destination detail, an arriving packet is moved to corresponding virtual channels (VCs) destination Virtual Output Queue.
Flow control may be used to avoid any packet drop in switches. Specifically, Credit-based flow control may be used to avoid any packet drops in switches. In credit-based flow control, packets are transmitted when (typically only when) buffer space is known to exist at the destination. To support credit-based flow control, a switch typically maintains counters for every destination per virtual channel (VC). This keeps track of buffer space at each destination D e.g. by decrementing the counter for every packet sent to destination, and incrementing the counter for every credit returned from destination D. Credits are returned from a destination whenever a destination buffer frees up. For a multi-VC switch, e.g. as shown in
Virtual Output Queues aka VOQ's typically store the incoming requests. A credit counter typically generates “ready” (a ready indication) whenever the destination has enough credit to accommodate a request. When “ready” is asserted, requests in all sources' Virtual Output Queue/s aka VOQ are typically exposed to the dispatch unit, else all requests are, typically, masked. Credit counters are per destination per virtual channel (VC) and are responsible for masking/unmasking of requests in corresponding virtual channels' (VCs′) corresponding destination Virtual Output Queue. This scheme is referred to as a “credit at dispatch” scheme.
In input queued switches, each input port can transmit at most 1 cell at a time. Each output port can receive at most 1 cell at a time and each output port can receive cells over only 1 virtual channel (VC) at a time. A dispatch unit may ensure that above constraints are met. In an N input, M output, and K VC switch, a dispatch unit may find a set of up to min (N,M) cells to transmit over the switch.
The Destination Choose Source (DCS), or first set of arbiters in the 2-D arbiter, typically provides each destination with a DCS, or DCS arbiter (aka “destination DCS”) to select among the sources requesting that destination. A Destination Choose Source (DCS) or “destination DCS” also typically includes VC select logic to arbitrate among the virtual channels (VCs) to that destination. Typically, the DCS arbiters of every destination are, functionally, exactly the same. For simplicity of explanation a single destination's DCS is now described, an example of which is illustrated in
The VC select logic (e.g. of
A DCS with DV arbiter in a round robin arbitration scheme and VC select logic either in a round robin arbitration scheme or least recently used scheme, is referred as a “Rolling-RR arbiter” (e.g. the “Rolling RR DCS” of
Referring again to
Typically, the dispatch unit of
Operation 1: Each input sends a request to at least one (typically each) output, to every virtual channel (VC) for which the input has at least one queued cell.
Operation 2: for each destination, each DV arbiter selects one source, from among the sources which have a request to this destination and virtual channel (VC). Typically, this results in one selected source per virtual channel (VC) per destination.
Operation 3: Each destination's VC select logic selects one of the DV arbiters' grants, from among the DV arbiters of this destination which have grants. Typically, this results in one selected virtual channel (VC) per destination.
Operation 4: Each source then arbitrates among those destination DCSs which have selected this source. Typically, this results in one selected destination per source.
It is appreciated that input queued switches may use any suitable crediting scheme to support credit-based flow control, such as but not limited to the following:
“Pre-creditor” scheme: in this scheme incoming requests are pre credited by deducting the destination credit and Dispatcher unit dispatches the pre credited requests e.g. as shown in
“Credit at dispatch”: Credits are deducted only for dispatched request. Requests are placed to dispatch unit when destination has enough credit to support at least one request e.g. as shown in
The existing Dispatch unit (e.g.
RR-LRU arbitration scheme: DCS arbiter uses Round Robin (RR) arbitration scheme and SCD arbiter uses Least Recently Used (LRU) scheme.
LRU-LRU arbitration scheme: DCS arbiter uses Least Recently Used (LRU) arbitration scheme and SCD arbiter uses Least Recently Used (LRU) scheme.
It is appreciated that existing switches may use the following combinations of crediting scheme and dispatch unit arbitration scheme:
“Pre-creditor” with RR-LRU algorithm—which may have a area/power penalty; or
“Credit at dispatch” with LRU-LRU algorithm—which may have a performance penalty.
Conventionally, a “Credit at dispatch” scheme, with an RR-LRU arbitration scheme, has performance, power and area benefit but may have a virtual channel (VC) starvation issue i.e. there may be VC request/s which remain unserved indefinitely. The dispatch unit described herein, e.g. the Rolling RR DCS-SCD illustrated in
An example of starvation may be appreciated by considering by way of example the 2×3 switch with 2 virtual channels (VCs) shown in
The example of starvation shown in
S0VC0 and S1VC0, belong to the same virtual channel, say VC0.
S0VC1 and S1VC1, belong to the same virtual channel, say VC1.
A round robin pointer moves in the S0VC0->S0VC1->S1VC0->S1VC1->S0VC0 . . . direction. If VC0 credit is available only after 3 clock cycles have elapsed from the time the VC0 credit is consumed, and if VC1 credit is always available, then D0 ARB may grant the request in the following order; the clock cycles 1-6 below may be seen in the top left, top right, middle left, middle right, bottom left and bottom right portions, respectively, of
Clock cycle 1: Both VC0 and VC1 credits are available, and all 4 requests are exposed to ARB. ARB pointer is at S0VC0 and the ARB grants request S0VC0. ARB pointer is updated to favor the request which is just after the one granted, i.e. S0VC1. Grant of (or responsive to) request S0VC0 consumes VC0 credit.
Clock cycle 2: VC0 credit is not available and all VC0 requests are masked. ARB grants request S0VC1. ARB pointer is updated to point to next request S1VC0. Grant of request S0VC1 consumes VC1 credits.
Clock cycle 3: ARB sees request from only S0VC1 and S1VC1. ARB pointer will skip S1VC0 since the request is masked by credit unavailability and, instead, ARB grants request S1VC1. ARB pointer is updated to point to next request i.e. request S0VC0.
Clock cycle 4: VC0 credit is returned, and ARB sees all 4 requests. Since ARB pointer is at request S0VC0 the ARB grants request S0VC0. ARB pointer is updated to point to the next request which is S0VC1. Grant of request S0VC0 consumes VC0 credits.
Clock cycle 5: VC0 credit not available due to grant to S0VC0 in previous clock cycle, as occurred in clock cycle 2. The ARB grants request S0VC1 and ARB pointer is updated to favor the next request.
Clock cycle 6: VC0 credit is not available, and ARB sees requests from only S0VC1 and S1VC1 as occurred in clock cycle 3. The ARB pointer will skip S1VC0 since the request is masked by credit unavailability, and, instead, the ARB grants request S1VC1.
Unfortunately, the above scenario can repeat ad infinitum—causing request S1VC0 to be “starved” i.e. to remain unserved indefinitely.
However, according to certain embodiments, a “credit at dispatch” scheme is used by masking a request when credit not available to ensure no request is granted other than when credit is available. The masking feature of the “credit at dispatch” scheme, and the skipping that occurs in the round robin aka RR algorithm, yield virtual channel victimization. However, certain embodiments herein solve the virtual channel victimization problem by decoupling the masking feature of the credit at dispatch scheme from the skipping that occurs in round robin algorithms i.e. the embodiment may freeze a round robin pointer corresponding to virtual channel request/s which is/are (e.g. currently) masked by the “credit at dispatch” scheme. This may be achieved by splitting a single round robin arbiter into plural virtual channel specific round robin arbiters, or by replacing a single round robin arbiter with plural round robin arbiters, each of which is virtual channel-specific, in combination with virtual channel select logic to select one of the DV arbiter grant vectors, to avoid two grants to the same destination (Rolling RR DCS).
Still referring to the above presented virtual channel victimization scenario with RR arbiter replaced by Rolling-RR DCS arbiter, assume for simplicity that the virtual channel select logic is simple LRU. The clock cycles 1-4 below may be seen in the top left, top right, bottom left and bottom right quadrants, respectively, of
Clock cycle 1: Both VC0 and VC1 credits are available, D0VC0 ARB sees request from S0VC0 and S1VC0, D0VC1 arbiter sees requests from S0VC1 and S1VC1. D0VC0 ARB grants request S0VC0 and D0VC1 arb grants request S0VC1. VC select logic selects or accepts grants from D0VC0 ARB. Virtual channel select logic pointer is updated to point to D0VC1 ARB. DV ARB pointer is updated if, and only if the grant is accepted by virtual channel select logic. In this case only VC0 ARB pointer is updated to point to next request i.e. request S1VC0 and D0VC1 ARB pointer are unchanged. Grant to request S0VC0 consumes VC0 credit.
Clock cycle 2: VC0 credit is not available due to grant to S0VC0 in previous clock cycle; all VC0 requests are masked. As a result, D0VC0 ARB sees no request. the pointer of D0VC1 ARB (which sees requests from S0VC1 and S1VC1) is at request S0VC1 so D0VC1 ARB grants request S0VC1. The VC select logic pointer is at D0VC1 thus the D0VC1 grants are accepted. VC select logic pointer is updated to favor D0VC0 ARB. The D0VC1 ARB pointer is updated to favor request S1VC1. The grant of request S0VC1 consumes VC1 credits.
Clock cycle 3: VC0 credit has not yet returned thus D0VC0 ARB sees no request. D0VC1 ARB sees requests from S0VC1 and S1VC1. D0VC1 ARB grants request S1VC1. VC select logic pointer is at D0VC0 but since there is no grant from D0VC0, the VC select logic accepts grants from D0VC1. The D0VC1 ARB pointer is updated to favor request S0VC1.
Clock cycle 4: VC0 credit is returned. The D0VC0 ARB sees requests S0VC0 and S1VC0. The D0VC1 ARB sees requests S0VC1 and S1VC1. The D0VC0 ARB grants request S1VC0 and D0VC1 ARB grants request S0VC1. VC select logic pointer, which is at D0VC0, accept grants from D0VC0 ARB. The VC select logic pointer is updated to favor D0VC1 ARB grants. The D0VC0 ARB pointer is updated to favor request S0VC0.
Thus in the above example, all requests are granted in 4 cycles, and zero requests are skipped, thereby to eliminate starvation.
The virtual channel select logic typically decides which DV ARB grants need to be considered. Having conventional or simple LRU logic with a grant pointer updated for every grant from DV ARB will lead to unfairness between virtual channels, e.g. as shown in
Area: Pre-creditor vs Credit at dispatch: The table of
These equations may be used for computation of credit:
Credit at dispatch
RTT+MTU
Pre creditor
RTT+MTU+ready queue depth+Extra credit (EC)
Where EC=(#src-1)*MTU
Performance: RR-LRU vs LRU-LRU: The graph of
From the graph (ARB efficiency RR vs LRU) of
The term “all” is used herein for simplicity, to describe example embodiments. It is appreciated however that alternatively, whatever is said herein to be true of or to characterize or to pertain to, “all” members of, or “each” member of, or “every” member of, a certain set can also, in other embodiments, be true of, or characterize or pertain to, most but not all members of that set, or all but a few members of that set, or at least one (but less than all) member/s of the set.
For example, a scheme may mask “all” requests, as described herein. But alternatively, most but not all requests, or all but a few requests, or plural requests, but less than all, may be masked. To give another example, flow controls are used to prevent any packet drop within switches as described herein. But alternatively, flow controls may be used to prevent most but not all packet drop within switches, or all but a few packet drop within switches, or at least one (but less than all) packet drop/s within switches.
It is appreciated that software components of the present invention may, if desired, be implemented in ROM (read only memory) form. The software components may, generally, be implemented in firmware or hardware, if desired, using conventional techniques. It is further appreciated that the software components may be instantiated, for example as a computer program product, or on a tangible medium. In some cases, it may be possible to instantiate the software components as a signal interpretable by an appropriate computer, although such an instantiation may be excluded in certain embodiments of the present invention.
It is appreciated that various features of the invention which are, for clarity, described in the contexts of separate embodiments may also be provided in combination in a single embodiment. Conversely, various features of the invention which are, for brevity, described in the context of a single embodiment, may also be provided separately, or in any suitable sub-combination.
It will be appreciated by persons skilled in the art that the present invention is not limited by what has been particularly shown and described hereinabove. Rather the scope of the invention includes, inter alia, the appended claims and equivalents thereof.
Number | Date | Country | Kind |
---|---|---|---|
202111040728 | Sep 2021 | IN | national |