Patent Application
20170075838
Aspects disclosed herein relate to the field of bus interconnects (also referred to herein as a “bus” or an “interconnect”). More specifically, aspects disclosed herein provide quality of service in bus interconnects with multi-stage (also referred to as multi-level) arbitration.
Components in integrated circuits, such as processors, memory, and caches, are conventionally connected via a bus interconnect. Modern interconnects include data transmissions paths in multiple dimensions (such as the x-, y-, and z-dimensions), and routing packets from a source to its destination along the interconnect involves arbitration in various stages by a number of different arbiters. Applying fair arbitration schemes at each stage is not fair to all packets. The average latencies for packets are high where fair arbitration is applied at each arbiter. The fair arbitration schemes may therefore lead to sub-optimal utilization of the bus, and do not permit the application of quality of service (QoS) to packets using the bus. However, any solutions to apply QoS to the bus interconnect must provide priority propagation that is low cost and as simple as possible.
Aspects disclosed herein provide quality of service (QoS) in bus interconnects with multi-level and/or multi-stage arbitration by tagging packets with an indication of priority. In at least one aspect, the indication of priority is based at least in part on a distance from packet source to packet destination.
In one aspect, an integrated circuit comprises a first arbiter for controlling access to a bus interconnect. The first arbiter is configured to receive a first packet having a first priority at a first input port, and receive a second packet having a second priority, wherein the first and second priorities are based on a distance to a respective destination of the first and second packets. The first arbiter is further configured to forward the first packet upon determining that the first priority is of a higher relative priority than the second priority.
In another aspect, a method comprises receiving a first packet having a first priority at a first arbiter configured to control access to a bus interconnect. The method further comprises receiving, by the first arbiter, a second packet having a second priority, wherein the first and second priorities are based on a distance to a respective destination of the first and second packets. The method further comprises forwarding, by the first arbiter, the first packet upon determining that the first priority is of a higher relative priority than the second priority.
In another aspect, an apparatus comprises means for receiving a first packet having a first priority. The apparatus further comprises means for receiving a second packet having a second priority, wherein the first and second priorities are based on a distance to a respective destination of the first and second packets. The apparatus may further comprise means for forwarding the first packet upon determining that the first priority is of a higher relative priority than the second priority.
In another aspect, an apparatus comprises a first computing element configured to generate a first packet specifying the first computing element as a source and a second computing element as a destination. The apparatus further comprises a first interface coupled to the first computing element. The interface may be configured to identify, in a routing table of the first interface, a priority for the first packet, wherein the priority is based on a distance from the first computing element to the second computing element, and insert an indication of the priority in the first packet.
In another aspect, a method comprises generating, by a first computing element, a first packet specifying the first computing element as a source and a second computing element as a destination. The method further comprises identifying, in a routing table of a first interface coupled to the first computing element, a priority for the first packet, wherein the priority is based on a distance from the first computing element to the second computing element. The method further comprises inserting, by the first interface, an indication of the priority in the first packet.
In still another aspect, an apparatus comprises means for generating a first packet specifying a first computing element, of a plurality of computing elements connected via a bus interconnect, as a source and a second computing element of the plurality of computing elements as a destination. The apparatus further comprises means for identifying a priority for the first packet, wherein the priority is based on a distance from the first computing element to the second computing element. The apparatus further comprises means for inserting an indication of the priority in the first packet.
By tagging packets with an indication of priority, aspects disclosed herein provide quality of service in bus interconnects. The indication of priority may be determined by source computing elements using lookup tables that are based on distances to destination computing elements. Doing so provides low cost propagation of priority in the bus interconnect. Arbiters in the bus interconnect may serve packets having indications of higher priority before serving packets having lower indications of priority. Doing so may provide more efficient utilization of the bus interconnect.
So that the manner in which the above recited aspects are attained and can be understood in detail, a more particular description of aspects of the disclosure, briefly summarized above, may be had by reference to the appended drawings.
It is to be noted, however, that the appended drawings illustrate only aspects of this disclosure and are therefore not to be considered limiting of its scope, for the disclosure may admit to other aspects.
Aspects disclosed herein provide quality of service in bus interconnects that perform multi-stage arbitration by tagging packets with an indication of priority that is based on a distance to the destination of each packet. A plurality of source computing elements are communicably coupled by the bus interconnect. Arbiters regulating access to the bus interconnect may use the indication of priority to serve packets having a higher relative priority before serving packets that have a lower relative priority. Generally, aspects disclosed herein provide at least two schemes for tagging packet with an indication of priority, namely priority classes and credits.
Under the priority class scheme, two or more priority classes may be defined. The priority classes may be based on the distance from a source computing element to a destination computing element in an integrated circuit, such as a system on a chip (SoC). As used herein, a “computing element” refers to any hardware structure that is connected to a bus. Examples of computing elements include, without limitation, processors (CPUs), memory, caches, peripherals, network interfaces, and the like. The distance between computing elements may be determined based at least one of: the number of arbiters along the path between the source and destination, and a number of cycles required for the packet to travel from the source to destination. Each interface connecting a given computing element to the bus may include lookup tables (also referred to as routing tables) that specify priority classes for each destination computing element. In at least one aspect, the priority class assigned to a packet is static, in that the packet retains the same priority class from source to destination.
Therefore, when a source computing element generates a first packet under the priority class scheme, the interface connecting the source to the bus interconnect may reference the lookup tables to determine a priority class for the first packet based on the packet's destination (a computing element). The interface may then insert an indication of the priority classes in the first packet. A first arbiter controlling access to the bus interconnect may receive the first packet specifying the first priority class at a first input port, and a second input packet specifying a second priority class at a second input port. The first and second priority classes are based on a distance to a respective destination of the first and second packets, respectively. The first arbiter may compare the first and second priority classes, and forward the first packet based on a determination that the first priority class is of a higher relative priority than the second priority class.
Under the credit-based scheme, the source interfaces tag packets with an indication of a number of credits. The number of credits may be based on a function of one or more of a distance from the source computing element to the destination computing element, a bandwidth of the bus interconnect, and a latency of the bus interconnect. In at least one aspect, the source interfaces store lookup tables that specify the number of credits that should be given to a packet based on the packet's destination. When receiving a packet from a source computing element, the source interface may reference the lookup table to determine how many credits the packet should receive. The source interface may then insert an indication of the number of credits into the packet. Each credit-based arbiter along the bus interconnect serves packets with higher numbers of credits before serving packets with lower numbers of credits. When serving a packet, the credit-based arbiters may decrement the number of credits of the packet, and in some aspects by a predefined number (that may be specific to an arbiter). Once decremented, an indication of the updated number of credits may then be stored in the packet.
Therefore, when a source computing element generates a first packet under the credit-based scheme, the source interface may reference the lookup tables to determine a number of credits for the first packet based on the packet's destination computing element. The source interface may then insert an indication of the number of credits in the first packet. A first arbiter controlling access to the bus interconnect may receive the first packet at a first input port, and a second input packet specifying a second number of credits at a second input port. The first and second numbers of credits are based on a distance to a respective destination of the first and second packets, respectively. The first arbiter may compare the first and second numbers of credits, and forward the first packet based on a determination that the first number of credits is of a higher relative priority than the second number of credits.
In addition, aspects disclosed herein provide different lookup tables for each phase of a bus transaction. For example, if a bus transaction includes a request phase, data phase, snoop phase, and a response phase, the source interfaces may include a lookup table for each phase of the transaction. The source interfaces may then reference the lookup table corresponding to the current phase of the bus transaction, and tag the packet with an indication of priority (e.g., credits or priority class).
In addition, aspects disclosed herein may assign separate indications of priority per dimension traveled from source to destination. Generally, modern bus interconnects are built in many dimensions. For example, a three-dimensional bus interconnect may be considered to have an x-dimension, a y-dimension, and a z-dimension. In such an example, a packet may be tagged with an indication of priority for each of the three dimensions, where each indication of priority is based on the distance (in each respective dimension) from the packet source to the packet destination. Therefore, the priority for the x-dimension would be based on the distance to the destination in the x-dimension, while the priority for the y-dimension would be based on the distance to the destination in the y-dimension, and the priority for the z-dimension would be based on the distance to the destination in the z-dimension. The source interfaces may therefore include lookup tables that specify priority classes and/or credits for each dimension in the bus interconnect.
Further still, aspects disclosed herein may provide the ability to map between the credit-based and priority class-based schemes. For example, a router (that includes multiple arbiters) may receive a packet that specifies a number of credits. The router may include logic to convert the number of credits to a priority class, such that a priority-class based arbiter in the router can serve the packet based on the converted priority class. Similarly, the router may include logic to convert a priority class to a number of credits.
Aspects of the disclosure are equally applicable to multiple types of bus interconnects, including all ring bus structures, mesh interconnects, and network on chip (NoC) interconnects. The use of any particular bus type herein as a reference example should not be considered limiting of the disclosure.
As shown, the computing elements 1011-9 are connected to the bus 104 via a respective source interface 1021-9. Each source interface 1021-9 includes logic (not pictured) configured to tag packets generated (or forwarded) by the computing elements 1011-9 with an indication of priority. Examples of source interfaces include bridges, where the bridge connects a given computing element 1011-9 to the bus 104. Each source interface 1021-9 may insert an indication of priority in packets based on priority information stored in one or more lookup tables of the source interface 1021-9. The indication of priority (e.g., a priority class or number of credits) may be inserted using hardware, software, firmware, or any combination thereof. Generally, the priority information stored in the lookup tables is based on a distance from the corresponding source computing element 1011-9 to a respective destination computing element 1011-9.
As shown, table 120 defines two example priority classes that are based on a number of arbiters (or routers 103) disposed between a source computing element 1011-9 and a destination computing element 1011-9. However, any number of priority classes may be defined. Similarly, the priority classes may be based on an absolute value (e.g., 1 arbiter) or range of values (e.g., 0-2 arbiters). As shown, the table 120 reflects a priority class indication of “0” where one or fewer routers 1031-9 are disposed on a path between the source computing element 1011-9 and a destination computing element 1011-9. Similarly, the table 120 reflects a priority class indication of “1” where more than one, but less than or equal to three routers 103 are disposed between the source computing element 1011-9 and a destination computing element 1011-9. In at least one aspect, since priority class 1 is associated with a greater distance between computing elements 101 than priority class 0, an arbiter in the routers 103 may serve packets tagged with priority class 1 before serving packets tagged with priority class 0, as priority class 1 is of a higher relative priority than priority class 0.
The example lookup table 110 includes values that are computed based on the topology of the integrated circuit 100 from the perspective of computing element 1011. Therefore, the lookup table 110 may be stored in the source interface 1021. However, in some aspects, the lookup table may be stored in the computing elements 101, which may include logic implemented by the interfaces 102 to tag packets with an indication of priority, such as a priority class or number of credits. As shown, the lookup table 110 includes a destination column 111, a column 112 for the x-dimension of the bus 104 (or east/west direction), and a column 113 for the y-dimension of the bus 104 (or the north/south direction). The destination column 111 specifies a computing element 101 that is a destination of packets sent by the computing element 1011. The columns 112 and 113 define values used by the interface 102 to tag packets generated (or forwarded) by the computing element 1011 with an appropriate priority class that is based on the distance to the respective destination computing element 1012-9. The values in the columns 112, 113 are shown with an example format that corresponds to “direction:distance:priority class”. However, the columns 112, 113 may take any format suitable to specify at least an indication of priority for a packet (such as a priority class or number of credits). The “direction” component may be a binary value that indicates whether to forward the packet via an east or west port of the interface 1021 (similarly, when added to a packet, this value may be used by the routers 103 to forward packets in the appropriate direction). The “distance” component may specify the distance (measured in number of routers 103) from the source computing element 1011 to a destination computing element 1012-9. The “priority” component specifies one of the two priority classes defined in the table 120.
Therefore, as shown, column 112 for destination 1013 specifies an example entry of “1:2:1”. The entry indicates that the interface 1021, when identifying a packet generated (or forwarded) by source 1011, should forward the packet through the east port of the interface 1021, that two routers/arbiters in the x-dimension are disposed between source 1011 and destination 1013 (for example, routers 1034 and 1035), and the packet should be tagged with priority class 1 as an indication of priority in the x-dimension. Similarly, as shown, the entry in column 113 for destination 1013 is “1:1:0”. This entry reflects that computing element 1013 is north of computing element 1011, that one router/arbiter in the y-dimension is disposed between source 1011 and destination 1013 (for example, router 1032), and the packet should be tagged with priority class 0 as an indication of priority in the y-dimension based on the number of routers 103 in the y-dimension between the source 1011 and destination 1013. Therefore, as shown, a given packet may have different priorities for the x-dimension and the y-dimension of the bus 104.
As each router 103 receives a packet 131, 132, the arbiters of the router 103 perform arbitration operations to allow one packet to access the bus 104 per cycle. The arbiters may compare priority classes in the packets to determine which packet to serve first. For example, assuming packets 131, 132 are in different input queues of an arbiter in router 1036, the arbiter may compare the priority classes specified in the packets 131, 132. The arbiter may then determine that packet 131, specifying a priority class of 1, has a higher relative priority than packet 132, which specifies a priority class of 0. As shown, therefore, the arbiter of router 1036 serves packet 131 before packet 132, forwarding packet 131 to interface 1026 (and subsequently to the destination computing element 1016). Packet 132 may be subject to one or more additional arbitration operations before the router 1036 forwards the packet 132 to its destination.
As shown, for example, lookup table 140 provides packets targeting computing element 1019 with 0 credits in the x-dimension and 1 credit in the y-dimension. Therefore, when source interface 1021 receives a packet specifying computing element 1011 as the source and specifying computing element 1019 as a destination, logic in the source interface 1021 may reference the table 140 using computing element 1019 as an index value. The logic may determine that the entry for computing element 1019 provides 0 credits in the x-dimension, and 1 credit in the y-dimension. Therefore, the logic in the source interface 1021 may insert, in the packet, an indication of 0 credits in the x-dimension and an indication of 1 credit in the y-dimension. In at least one aspect, the logic in the source interface 1021 may sum the credits in the x-dimension and y-dimension and insert an indication of the summed number of credits in the packet. Logic in the source interface 1021 may then forward the packet onto the bus 104.
To implement the credit-based scheme, the arbiters of the routers 103 include logic configured to perform arbitration operations based on the number of credits specified in each packet. The arbiters may compare the numbers of credits of each packet in the input queues of the arbiter, and serve the packet having the highest number of credits (e.g., by forwarding the packet via an output port of the arbiter). The arbiters (or the routers 103) may include logic (not pictured) to decrement the credits of the packets by a predefined number of credits as packets are served. The predefined number of credits may be specific to each arbiter/router, and may be based on the complexity of the arbiter/router. For example, router 1034 may decrement 2 credits, router 1035 may decrement 1 credit. Before forwarding the packet via an output port of the arbiter, the logic may update the number of credits in the packet to reflect the number of credits that were decremented.
In at least one aspect, the routers 1031-9 may include both credit-based and priority class-based arbiters. Further still, some routers 1031-9 may include only credit-based arbiters, while other routers 1031-9 include only priority-class based arbiters. To support such hybrid deployments, the routers 1031-9 may include logic to convert credits to priority classes, and convert priority classes to credits.
Further still, the lookup tables 110, 140 may be specific to one or more phases (of multiple phases) of a bus transaction. The source interfaces 1021-9 (and/or the computing elements 1011-9) may therefore have a lookup table for each phase of a bus transaction. For example, a bus transaction may have, without limitation, a command phase, a data phase, a response phase, and a snoop phase. Therefore, in such aspects, the source interfaces 1021-9 may have a command phase lookup table, a data phase lookup table, a response phase lookup table, and a snoop phase lookup table. Additionally, a given lookup table may provide different schemes for different dimensions. For example, lookup table 110 may specify credits for the x-dimension, and priority classes for the y-dimension.
The priority manager 201 is logic configured to insert an indication of priority into packets generated (or forwarded) by a computing element 101. The indications of priority may be based on predefined formatting, and inserted in predefined locations in each packet. When a source computing element 101 attempts to transmit a packet via the bus 104 as part of a phase of the bus transaction, the priority manager 201 in the source interface 102 may reference the lookup table 202 corresponding to the phase of the bus transaction. The priority manager 201 may use a destination of the packet as an index into the appropriate lookup table 202. The priority manager 201 may then determine the number of credits and/or the priority class associated with the destination, and insert an indication of the number of credits and/or the priority class in the packet. The priority manager 201 may then forward the packet for arbitration in multiple stages by the arbiters 205. The priority manager 201 may be implemented using hardware, software, firmware, or any combination thereof.
The arbiters 205 are devices configured to control access to a shared resource, such as the bus 104. In some aspects, the arbiters 205 include priority arbiters that provide quality of service (QoS) by serving packets based on indications of priority stored in the packets, such as credits or priority classes. The routers 103 may include arbiters 205 that are configured to arbitrate based on priority class and arbiters 205 that are configured to arbitrate based on credits. The arbiters 205 may include logic (hardware, software, or firmware) that can determine whether to update the number of credits in a packet (or the priority class of a packet) that is served by an arbiter 205. Examples of such logic include incrementing logic and decrementing logic. Similarly, the arbiter 205 may include logic that does not modify the priority class or number of credits of a packet. In at least one aspect, such logic is included in the arbitration manager 204 of the routers 103.
Similarly, the arbitration logic 2120-N may compare numbers of credits stored in packets, and serve the packet having the greatest number of credits. For example, if packet x in input port 2100 specifies an indication of 1 credit, while packet y in input port 210N specifies an indication of 2 credits, the arbitration logic 2120-N may grant packet y access to the corresponding output port 2130-N before granting packet x access to the output port 2130-N based on a comparison of the credits (where greater numbers of credits indicate higher relative priorities). In the event of a tie (e.g., the numbers of credits are equal), the arbitration logic 2120-N performs fair arbitration.
When granted access to an output port 2130-N, a packet may then traverse the bus 104 for another stage of arbitration by arbiters 205 in a different router 103, or the packet may arrive at a destination computing element 101.
As shown, the method 300 begins at step 310, described in greater detail with reference to
At step 330, logic is provided at the source interfaces to tag packets with a k-bit indication of priority. The logic is configured to determine the k-bit priority by referencing the lookup tables using a destination of the packet as an index. The logic may then insert the corresponding k-bit priority in the packet. The k-bit indication of priority may correspond to a priority class and/or a number of credits. At step 340, described in greater detail with reference to
As shown, the method 400 begins at step 410, where at least two priority classes are defined. Example priority classes are depicted in table 120 of
At step 420, a loop including steps 430-490 is performed to build a priority class-based lookup table for each computing element as a source computing element in the integrated circuit. At step 430, a loop including steps 440-470 is performed for each destination computing element relative to the current computing element as a source. At step 440, the distance from the current source computing element to the current destination computing element is determined. The distance may include distances in each dimension of the bus interconnect, such as an x-dimension, y-dimension, and z-dimension in a three-dimensional interconnect. The distances may be determined based on a number of arbiters between the computing elements and/or a number of cycles required for a packet to travel from the source to the destination.
At step 450, a priority class may be determined for each dimension of the interconnect based on the distance from the source to the destination. For example, continuing with the above priority classes for the request phase (based on number of arbiters), if the distance in the x-dimension is 1 arbiter, the distance in the y-dimension is 2 arbiters, and the distance in the z-dimension is four arbiters, an entry in the lookup table for the current source to the current destination would specify the second priority class (zero to two arbiters) in the x-dimension and the y-dimension, and the third priority class (three or more arbiters) in the z-dimension. When the source computing element subsequently generates a packet targeting the current destination as part of a request phase of a bus transaction, the priority manager 201 (or the source itself) may tag the packet with indications of the priority classes in each dimension. Using the determined distances against the defined priority classes for each phase of the bus transaction, entries for the lookup table for each phase of the bus transaction may be defined.
In some aspects, however, a common priority class may be specified for each dimension based on the total number of arbiters in each dimension. In such aspects, the lookup tables may specify ranges and/or absolute values for the total number of arbiters in all dimensions, and the priority manager 201 may tag packets with an indication of a single priority class.
At step 460, an indication of the priority classes for the current source to the current destination determined at step 450 may be stored in the lookup tables of the interface of the current source computing element. As previously indicated, priority classes for each dimension of the bus interconnect may be computed at step 450. Therefore, in such aspects, the computed priority classes for each dimension may be stored in a lookup table for the respective dimension of the bus interconnect at step 460 (where the interface stores a lookup table for each dimension of the bus interconnect). Step 470 of the method 400 includes a determination as to whether more destination computing elements remain relative to the current source computing element. If more destination computing elements remain, the method returns to step 430. If no more destination computing elements remain, priority classes for each destination computing element relative to the current source computing element have been defined, and the method proceeds to step 480. At step 480, the priority-class based lookup tables may be stored in the source interface 102 (and/or the computing element 101) as lookup tables 202. Step 490 of the method 400 includes a determination of whether more source computing elements in the integrated circuit remain. If more source computing elements remain, the method 400 returns to step 420 to build priority-class based lookup tables for the remaining source computing elements. Otherwise, the method 400 ends.
At step 510, a user may define at least one function to compute credits based on one or more of the distance from a source to a destination computing element, bandwidth of the bus interconnect, and latency of the bus interconnect. In at least one aspect, different functions may be defined for each phase of a bus transaction. Doing so may facilitate allocation of more credits to more important phases of the bus transaction, and fewer credits to less important phases of the bus transaction. At step 520, the bandwidth and/or latency of the bus interconnect may optionally be determined. The bandwidth of the bus interconnect may be determined by computing a product of the width of the bus and the speed of the bus. The latency of the bus may be determined based on computer-based simulations, or may be based on historic latency values for similar bus interconnects.
At step 530, a loop including steps 540-590 is performed to build a credits-based lookup table for each computing element as a source computing element in the integrated circuit. At step 540, a loop including steps 550-580 is performed for each destination computing element relative to the current computing element as a source. At step 550, the distance from the current source computing element to the current destination computing element is determined. The distance may include distances in each dimension of the bus interconnect, such as an x-dimension, y-dimension, and z-dimension in a three-dimensional interconnect. The distances may be determined based on a number of arbiters between the computing elements and/or a number of cycles required for a packet to travel from the source to the destination.
At step 560, one of the functions defined at step 510 may be applied to compute a number of credits for the current source computing element relative to the current destination computing element. In at least one aspect, the functions for each phase of the bus transaction may be applied to determine the respective number of credits for each phase of the bus transaction. In one aspect, the number of credits may be a total number of credits for all dimensions of the bus interconnect. In such an aspect, the priority manager 201 may tag a packet with an indication of the total number of credits. In other aspects, the functions may be invoked using the distances in each dimension to compute a number of credits for each dimension in the interconnect. At step 570, an indication of the computed numbers of credits may be stored in the credit-based lookup table for the current source computing element.
Step 580 of the method 500 includes a determination as to whether more destination computing elements remain relative to the current source computing element. If more destination computing elements remain, the method returns to step 540. If no more destination computing elements remain, credits for each destination computing element relative to the current source computing element have been computed, and the method proceeds to step 590. At step 590, the credit-based lookup tables may be stored in the source interface 102 (and/or the computing element 101) as lookup tables 202. Step 595 of the method 500 includes a determination of whether more source computing elements in the integrated circuit remain. If more source computing elements remain, the method 500 returns to step 530 to build credit-based lookup tables for the remaining source computing elements. Otherwise, the method 500 ends.
At step 630, the current arbiter 205 is configured to apply credit-based arbitration or priority class-based arbitration. While either scheme may be applied by any arbiter 205, in some aspects, the location of the arbiter 205 in the topology lends itself to favoring one scheme over the other. For example, for arbiters 205 (or routers 103) in the center of the topology, it may be more advantageous to configure the arbiters 205 to apply the priority class scheme. Similarly, for arbiters 205 (or routers 103) located near endpoints (e.g., computing elements 101), it may be more advantageous to apply the priority class scheme. As another example, the credit-based scheme may be more appropriate for intermediate arbiters 205 that are situated along the path between endpoints and the center of the topology. As previously indicated, however, some arbiters 205 in a given router 103 may apply the credit scheme, while other arbiters in the router 103 may apply the priority class scheme.
If the current arbiter 205 is configured as a credit-based arbiter, the method proceeds to step 640, where the current arbiter 205 is configured to compare indications of the number of credits in each packet in the input ports of the arbiter 205. The current arbiter 205 is further configured to serve the packet having the highest number of credits in a given cycle. In at least one aspect, the arbitration logic 212 of the arbiters 205 are configured to perform the comparison of credits and serve the input port having the packet with the highest number of credits. The arbitration logic 212 may further be configured to identify the credits for the appropriate dimension of the interconnect. For example, if the current arbiter 205 controls access to the y-dimension of the interconnect, the arbitration logic 212 may identify the y-dimension credits in the packet.
If the current arbiter 205 is a credit-based arbiter, the method proceeds to step 650, where the current arbiter 205 is configured to optionally change the number of credits of a packet that is served by the arbiter 205. For example, the current arbiter 205 may increment or decrement the number of credits of the packet by a predefined number of credits. The predefined number of credits may be based on the complexity and/or the location of the current arbiter 205 in the topology. The predefined number of credits may also be based on the dimension of the interconnect being controlled by the current arbiter 205. The arbitration logic 212 (and/or the arbitration manager 204) may be configured to decrement or increments the credits by the predefined number of credits, and store an indication of the updated number of credits in the packet once the credits have been changed. The arbitration logic 212 may further update the credits of the appropriate dimension, in aspects where credits for each dimension are specified in the packet. Similarly, the arbiter 205 may be configured to not modify the number of credits before forwarding the packet. As previously indicated, in changing the number of credits for the packet, the arbiter 205 may apply an algorithm to recompute a new number of credits for the packet in one or more dimensions, and store the updated numbers of credits in the packet. Once number of credits has optionally been changed, the method proceeds to step 670.
Returning to step 630, if the current arbiter 205 is a priority class-based arbiter, the method proceeds to step 660, where the current arbiter 205 may be configured to compare the indications of priority classes of the packets in each input port of the arbiter 205. The arbiter 205 may further be configured to serve the packet having the highest priority class in a given cycle. In at least one aspect, the arbitration logic 212 of the arbiter 205 is configured to perform the arbitration operations based on a comparison of priority classes, serving the packet with the highest priority class. In aspects where the packet is tagged with an indication of a priority class for each dimension of the interconnect, the arbitration logic 212 is configured to compare the priority classes corresponding to the dimension of the interconnect controlled by the current arbiter 205. At step 665, the current arbiter 205 may be configured to optionally change the priority class of served data packet. For example, the arbiter 205 may apply an algorithm to compute an updated priority class for the served data packet. The method then proceeds to step 670.
At step 670, the arbitration logic 212 of the current arbiter 205 is configured to perform fair arbitration in the event of ties between the priority of packets (e.g., numbers of credits or priority classes). At step 680, if more arbiters remain in the topology, the method returns to step 610. Otherwise, if no more arbiters remain, the method 600 ends.
At step 730, a loop including steps 740-790 is performed for each arbiter 205 (or router) disposed between the source computing element and the destination computing element of the first packet. At step 740, the current arbiter receives data packets, including the first data packet, in the input ports of the current arbiter 205. At step 750, the arbitration manger 204 may optionally convert credits to a priority class or convert a priority class to credits, as the case may be, to ensure that the first packet includes an indication of priority that can be used by the current arbiter as part of an arbitration operation.
Generally, each of the packets may target a respective output port of the current arbiter 205. Therefore, the current arbiter 205 must perform an arbitration operation to serve the packets and grant packets access to the output port one packet per cycle. The arbitration operation is based on the type of the current arbiter (credit based or priority class based). If the current arbiter 205 is a priority class-based arbiter, the method proceeds to step 760, where the current arbiter 205 serves the packet with the highest priority based on a comparison of the priority classes specified in each packet until the first packet is served, granting the first packet access to the output port of the current arbiter 205. In at least one aspect, the arbitration logic 212 of the arbiter 205 performs the arbitration operation by comparing the priority classes specified in each packet. The arbitration logic 212 may then determine which packet has the highest priority class, and grant the packet having the highest priority class access to the output port. At step 765, the current arbiter 205 may optionally modify the priority class of the first packet. For example, the current arbiter 205 may optionally increase or decrease the priority class of the current packet. The method may then proceed to step 790.
Returning to step 750, if the current arbiter is credit-based, the method proceeds to step 770, where the current arbiter 205 serves the packet with the highest priority based on a comparison of the number of credits in the packets until the first packet is served. In at least one aspect, the arbitration logic 212 of the arbiter 205 performs the arbitration operation by comparing the numbers of credits specified in each packet. The arbitration logic 212 may then determine which packet has the highest number of credits, and grant the packet having the highest number of credits access to the output port. At step 780, the credit-based arbiter 205 may decrement the first packet by a predefined number of credits. The number of credits decremented from the first packet may be a static number of credits specific to the current arbiter 205. In addition, the current arbiter 205 may recompute the number of credits for the current packet. For example, if the packet has zero credits (pre-decrementing or post-decrementing), the current arbiter may recomputed a number of credits for the packet based on distance to destination before forwarding the packet. The current arbiter 205 may also store an indication of an updated number of credits (decremented, unaltered, or recomputed) in the first packet before forwarding the packet on the bus 104. At step 790, if the first packet is received at a next arbiter, the method returns to step 730. If no more arbiters remain between the source and destination, at step 795, the first packet is received at its destination.
The computing device 801 generally includes the SOC 100, which includes a processor 804 connected via the bus 104 to a memory 808, a network interface device 818, a storage 809, an input device 822, and an output device 824. The computing device 801 is generally under the control of an operating system (not shown). Any operating system supporting the functions disclosed herein may be used. The processor 804 is included to be representative of a single CPU, multiple CPUs, a single CPU having multiple processing cores, and the like. The network interface device 818 may be any type of network communications device allowing the computing device 801 to communicate with other computing devices via the network 830.
The storage 809 may be a persistent storage device. Although the storage 809 is shown as a single unit, the storage 809 may be a combination of fixed and/or removable storage devices, such as fixed disc drives, solid state drives, SAN storage, NAS storage, removable memory cards or optical storage. The memory 808 and the storage 809 may be part of one virtual address space spanning multiple primary and secondary storage devices.
The input device 822 may be any device for providing input to the computing device 801. For example, a keyboard and/or a mouse may be used. The output device 824 may be any device for providing output to a user of the computing device 801. For example, the output device 824 may be any conventional display screen or set of speakers. Although shown separately from the input device 822, the output device 824 and input device 822 may be combined. For example, a display screen with an integrated touch-screen may be used.
As shown, the computing device 801 includes a plurality of interfaces 102 and routers 103, described in greater detail above. Generally, the interfaces 102 connect computing elements (such as the processor 804, memory 806, storage 808, network interface 818, input device 822, and output device 824) to the bus 104, and may tag packets generated by the computing elements with an indication of priority (e.g., credits and/or priority class). The routers 103 are generally configured to control access to the bus 104 by packets generated by the computing elements by providing arbiters 205 configured to compare the indications of priority in each packet.
Advantageously, aspects disclosed herein provide quality of service to bus interconnects implementing multi-stage arbitration in an integrated circuit such as a system on a chip (SoC). Generally, priority classes and/or credits are assigned to a packet at the source based on a distance to the destination of the packet. Advantageously, aspects disclosed herein provide the ability to assign separate priority classes and/or credits for each dimension of the bus interconnect. Furthermore, the source computing elements may include different lookup tables for each phase of a bus transaction. Advantageously, arbiters controlling access to the bus interconnect may apply either scheme, as aspects disclosed herein provide the ability to swap between schemes along the path traveled by packets.
A number of aspects have been described. However, various modifications to these aspects are possible, and the principles presented herein may be applied to other aspects as well. The various tasks of such methods may be implemented as sets of instructions executable by one or more arrays of logic elements, such as microprocessors, embedded controllers, or IP cores.
The various operations of methods described above may be performed by any suitable means capable of performing the operations, such as a processor, firmware, application specific integrated circuit (ASIC), gate logic/registers, memory controller, or a cache controller. Generally, any operations illustrated in the Figures may be performed by corresponding functional means capable of performing the operations.
The foregoing disclosed devices and functionalities may be designed and configured into computer files (e.g. RTL, GDSII, GERBER, etc.) stored on computer readable media. Some or all such files may be provided to fabrication handlers who fabricate devices based on such files. Resulting products include semiconductor wafers that are then cut into semiconductor die and packaged into a semiconductor chip. Some or all such files may be provided to fabrication handlers who configure fabrication equipment using the design data to fabricate the devices described herein. Resulting products formed from the computer files include semiconductor wafers that are then cut into semiconductor die (e.g., the integrated circuit 101) and packaged, and may be further integrated into products including, but not limited to, mobile phones, smart phones, laptops, netbooks, tablets, ultrabooks, desktop computers, digital video recorders, set-top boxes and any other devices where integrated circuits are used.
In one aspect, the computer files form a design structure including the circuits described above and shown in the Figures in the form of physical design layouts, schematics, a hardware-description language (e.g., Verilog, VHDL, etc.). For example, design structure may be a text file or a graphical representation of a circuit as described above and shown in the Figures. Design process preferably synthesizes (or translates) the circuits described below into a netlist, where the netlist is, for example, a list of wires, transistors, logic gates, control circuits, I/O, models, etc. that describes the connections to other elements and circuits in an integrated circuit design and recorded on at least one of machine readable medium. For example, the medium may be a storage medium such as a CD, a compact flash, other flash memory, or a hard-disk drive. In another aspect, the hardware, circuitry, and method described herein may be configured into computer files that simulate the function of the circuits described above and shown in the Figures when executed by a processor. These computer files may be used in circuitry simulation tools, schematic editors, or other software applications.
The implementations of aspects disclosed herein may also be tangibly embodied (for example, in tangible, computer-readable features of one or more computer-readable storage media as listed herein) as one or more sets of instructions executable by a machine including an array of logic elements (e.g., a processor, microprocessor, microcontroller, or other finite state machine). The term “computer-readable medium” may include any medium that can store or transfer information, including volatile, nonvolatile, removable, and non-removable storage media. Examples of a computer-readable medium include an electronic circuit, a semiconductor memory device, a ROM, a flash memory, an erasable ROM (EROM), a floppy diskette or other magnetic storage, a CD-ROM/DVD or other optical storage, a hard disk or any other medium which can be used to store the desired information, a fiber optic medium, a radio frequency (RF) link, or any other medium which can be used to carry the desired information and can be accessed. The computer data signal may include any signal that can propagate over a transmission medium such as electronic network channels, optical fibers, air, electromagnetic, RF links, etc. The code segments may be downloaded via computer networks such as the Internet or an intranet. In any case, the scope of the present disclosure should not be construed as limited by such aspects.
The previous description of the disclosed aspects is provided to enable a person skilled in the art to make or use the disclosed aspects. Various modifications to these aspects will be readily apparent to those skilled in the art, and the principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope possible consistent with the principles and novel features as defined by the following claims.
