Patent Application
20130042252
1. Field of the Invention
This invention relates to the field of integrated circuits. More particularly, this invention relates to the allocation within integrated circuits of processing resources for processing transactions received from a plurality of transaction sources.
2. Description of the Prior Art
It is known to provide within an integrated circuit a plurality of transaction sources, such as transaction masters, which generate transactions that are communicated via interconnect circuitry to request servicing circuitry, such as transaction slaves. The request servicing circuitry will normally contain finite processing resources for handling received transaction requests. If too many transaction requests are received, then insufficient processing resources will be available to handle those received transactions. Conversely, if a check is forced for the availability of processing resources before any transaction is sent, then this decreases efficiency and scalability as in many cases such processing resources will be available and the transaction could be successfully sent without this additional check.
Viewed from one aspect the present invention provides an integrated circuit comprising:
a plurality of transaction sources configured to generate transaction requests; and
request servicing circuitry configured to process said transaction requests using a set of processing resources; wherein
said request servicing circuitry is configured to switch between dynamic allocation of said processing resources among said transaction requests and static allocation of said processing resources among said transaction requests using a selection algorithm.
The present technique recognises that the request servicing circuitry may be configured to switch between dynamic allocation of processing resources in which the processing resources are allocated primarily on a first-come-first-served basis without prior reservation and static allocation in which processing resources are reserved for particular transactions and then those transactions permitted to proceed. The static allocation may be performed using a selection algorithm to select among the transaction requests generated by the plurality of transaction sources.
The selection algorithm may take many different forms and be dependent upon a large number of different parameters. In some embodiments the selection algorithm is dependent upon a respective priority level associated with a quality of service value for each of the plurality of transaction requests. In this way, it is possible for the request servicing circuitry to statically allocate processing resources preferentially to transaction requests associated with higher levels of quality of service in a manner which meets the desired performance goals of the integrated circuit.
The exchange of requests that may take place between the request servicing circuitry and the transaction sources can have a variety of different forms. In some embodiments,
said request servicing circuitry is configured to:
said requesting transaction source is configured:
said request servicing circuitry is configured:
Within transaction requests awaiting static allocation that share a particular of level of priority, the selection algorithm may be configured to use round robin selection as this will ensure that no individual transaction request will be starved of allocation as well as being relatively simple to implement and predictable in behaviour.
In some embodiments the selection algorithm may be formed so as to select between transaction requests awaiting static allocation that have different priority levels in dependence upon a starvation ratio. In this way, a transaction request of a lower priority level may be selected in place of a transaction request of a higher priority level if the starvation ratio for that lower priority level transaction request with respect to the higher priority level has been exceeded. Thus, for example, one lower priority level transaction request may be selected for every N higher priority level transaction requests. This starvation ratio may be made programmable so as to tune the behaviour of the system to the desired performance.
The transaction requests may each have a priority level within a hierarchy of priority levels extending from a lowest level to a highest level. In this context the request servicing circuitry may be configured to provide a maximum number of the processing resources that can be concurrently allocated to service transaction requests within each level of the hierarchy. Thus, lower priority levels may use up to a maximum number of the processing resources which is lower than the maximum number of processing resources that can be used by a higher priority level. The maximum number associated with each priority level may monotonically increase with the hierarchy of priority.
The number of processing resources allocated to transaction requests within a given level of hierarchy may be tracked by associated counters. These counters can be incremented when the request servicing circuitry allocates a processing resource to a transaction request at a given level and decremented when such a processing resource ceased to be allocated to a transaction request at that given level. Thus the maximum allocation limits may be tracked and enforced.
The counters may be incremented each time a resource is allocated or ceased to be allocated for the individual priority level concerned or in some embodiments for the individual priority level and all of the lower priority levels. This latter approach has the advantage of preserving processing resources to be available for higher priority levels.
The maximum number of processing resources that may be allocated to each priority level may in some embodiments be a programmable parameter. This allows the system to be adapted to the form of the different transaction sources and the nature of the processing workload in a manner which may be helpful to avoid particular transaction sources or processing operations being starved of allocation of processing resources.
The plurality of transaction sources can take a variety of different forms, such as a graphics processing unit, input/output coherent devices or in some embodiments each transaction source may comprise a processor cluster (e.g. multiple processor cores with a local cache memory).
The request servicing circuitry may again take a variety of different forms. In one particular form of the use this technique the request servicing circuitry may comprise a shared cache memory.
The integrated circuit may comprise a plurality of such request servicing circuitry each of these having a set of processing resources for allocation among the transaction requests and each managing its allocation in accordance with the above techniques.
The processing resources may take a variety of different forms as will be understood by those in this technical field. In one form of the present technique the processing resources may comprise a set of processing slots available to receive transaction requests.
The integrated circuit may include interconnect circuitry configured to communicate the transaction requests between the transaction sources and the request servicing circuitry. This interconnect circuitry can have a variety of different forms. One particular form of interconnect circuitry is a ring-based interconnect. Such ring-based interconnects are efficient to implement and scale as more transaction sources and request servicing circuitry is introduced into the integrated circuit.
Viewed from another aspect the present invention provides an integrated circuit comprising:
a plurality of transaction source means for generating transaction requests; and
request servicing means for processing said transaction requests using a set of processing resource means for processing; wherein
said request servicing means is configured to switch between dynamic allocation of said processing resource means among said transaction requests and static allocation of said processing resources among said transaction requests using a selection algorithm.
Viewed from a further aspect the present invention provides a method of communicating within an integrated circuit comprising the steps of:
generating transaction requests using a plurality of transaction sources;
processing said transaction requests using a set of processing resources; and
switching between dynamic allocation of said processing resource means among said transaction requests and static allocation of said processing resources among said transaction requests using a selection algorithm.
The above, and other objects, features and advantages of this invention will be apparent from the following detailed description of illustrative embodiments which is to be read in connection with the accompanying drawings.
The transaction sources 6, 8, 10, 12, 14, 16, 18, 20 can each generate transaction requests that have an associated quality of service value (added under software or hardware control) and which are transmitted around the ring-based interconnect 30 to the appropriate one of the request servicing circuits 22, 24. In the case of the shared cache memories 22, 24, the shared cache memories 22, 24 may be memory mapped to different regions of memory address space and accordingly each serve to service transaction requests addressed to the region of memory address space to which they are mapped. The respective POC/POSs 32, 34 can detect memory addresses associated with their shared cache memory 22, 24 and attempt to allocated processing resources to handling the received transaction requests as will be discussed below. Transaction requests which cannot be serviced by the shared cache memories 22, 24, either because they are outside the memory address space ranges concerned or because the memory address is missed within the shared cache memories 22, 24, may be sent to the memory controller 26 to be subject to generation of a further transaction out to the main memory 4. Some transaction requests may map to the input/output device 28 for servicing by this device.
Transaction requests that are issued on to the ring-based interconnect 30 will by default include a dynamic credit (a flag or signal indicating the proposed allocation mode) indicating that the POC/POS 32, 34 should attempt dynamic allocation of processing resources to that transaction request as a first option. If processing resources are not available for that transaction request, then a refusal response is returned to the transaction source concerned via the ring-based interconnect 30 and the transaction source placed into a state where it waits until it receives a proceed request before it resends the transaction request.
The POC/POS 32, 34 will send the proceed request when it has a processing resource available to allocate to that transaction request and will statically allocate (reserve for future use) that processing resource to the transaction source concerned before the proceed response is sent (the transaction source can use the statically allocated processing resource for the same transaction that cause the refusal response or a different pending transaction (e.g. a queued transaction request specifying a higher quality of service). Accordingly, when the transaction source receives the proceed request, this will effectively include a static credit indicating that it may resend its transaction request (or a different transaction request) including that static credit indication back to the POC/POS 32, 34 where a processing resource will already be allocated to that transaction source and which accordingly will be guaranteed to be accepted.
The transaction requests have a quality of service value associated with them. The POC/POSs 32, 34 map using a programmable mapping configuration between such quality of service values and a plurality of priority levels within a hierarchy of priority levels extending from a lowest level to a highest level (in some embodiments the mapping could be static/non-programmable). The POC/POSs 32, 34 are configured to allocate a maximum number of processing resources from the set of processing resources available to them to each priority level. Thus, a low priority level may have a small number of processing resources set as the maximum number which may be concurrently allocated to transaction requests at that priority level whereas a high priority level may have a higher number of processing resources set as the maximum number of processing resources that can be concurrently allocated to that higher priority level. These maximum numbers of processing resources which may be allocated can be fixed or programmable. The maximum number of processing resources monotonically increases as the priority level rises such that the higher priority levels have higher maximum numbers of processing resources that are potentially allocatable to the transaction requests having those priority levels.
When a POC/POS 32, 34 allocates a processing resource for a given priority level, it tracks this using a counter value. The counter is incremented each time a processing resource is allocated for that priority level and decremented each time a processing resource ceases to be allocated to that priority level. In some embodiments the counters for each priority level alone may be incremented and decremented when an allocation is changed as discussed above, but in other embodiments the increments and decrements are applied to the priority level concerned as well as to the counters of all the lower priority levels that are also being tracked. For example, in a system including a high high priority level, a high priority level, a medium priority level and a low priority level in a priority hierarchy, should a processing resource be allocated for the high high priority level, then the counter for the high priority level as well as the counter for the medium priority level and the low priority level will all be incremented. When a processing resource ceases to be allocated to a transaction request from the high priority level, then the counters for the high priority level, the medium priority level and the low priority level will all be decremented. In this way, the selection algorithm may be guided to exhaust the pool of processing resources that can be allocated to lower priority levels before the pool of resources that are allocatable to the higher priority levels are exhausted. It is also possible that these counters may individually track their own priority level without such inter-dependence.
Pending request counters within the POC/POSs 32, 34 are also provides to track the number of transaction requests of each priority level that are waiting to receive a proceed response as well as the identity of the transaction sources and the quality of service values of the refused requests concerned. These pending request counters are uses to track the number and details of the transaction requests awaiting static allocation (i.e. to be allocated a processing resource and sent proceed response (a static credit)).
When a plurality of transaction requests of a given priority level are awaiting allocation of a processing resource, the selection algorithm can select between these transactions using a round robin selection technique in which a pointer is used, and advanced with each selection, with a queue of transaction requests of that priority level awaiting allocation of a processing resource.
The selection algorithm may also select between transaction requests awaiting static allocation that have different priority levels in dependence upon a starvation ratio. It is desirable that a steady stream of transaction request of higher priority level should not completely prevent any transactions from a lower priority level from being allocated processing resources. Accordingly, starvation counters can be used to count how many times a transaction request is selected from a higher priority level in preference over a queued transaction request from a lower priority level. When this count reaches a starvation ratio level (which may be programmable) then a transaction request from the lower priority level is selected in preference to one from the higher priority level such that at least some of the transaction requests from the lower priority level make forward progress.
The transaction sources 6, 8, 10, 12, 14, 16, 18, 20 of
The programmable mapping configuration data can provide different mappings between the quality of service values and the priority values as illustrated in
In the different mapping illustrated to priority levels PL′, the quality of service values 0-8 are mapped to the priority level low, the quality of service values 8-12 are mapped to the priority level medium, the quality of service values 13-14 are mapped to the priority level high and the quality of service value 15 is mapped to the priority level high high.
If the determination at step 52 is that the transaction request is seeking a dynamic credit, then step 56 determines whether the number of processing resources already allocated to the priority level of the transaction request received at step 50 is already equal to or greater than the maximum number of processing resources that may be allocated to that level. If this maximum number has not be exceeded, then step 58 serves to dynamically allocate one of the processing resources to process the received transaction and a response indicating such a dynamic allocation has been successful is returned to the transaction source for the transaction request received at step 50. The counter value(s) tracking the allocation of processing resources are again incremented. These counter value(s) are decremented when a transaction ceases to use an allocated resource and it is released to be allocated to a different transaction.
If the determination at step 56 is that the transaction request that has been received requesting a dynamic credit cannot be dynamically allocated a processing resource since the maximum number permitted for that priority level has been exceeded, then processing proceeds to step 60 where a refusal response is returned to the requesting transaction source. Step 62 then increments the count value for the priority level concerned to indicate that one more transaction request is awaiting allocation of a processing resource for that priority level.
When a processing resource becomes available for static scheduling at step 66, processing proceeds to step 68 where a determination is made as to whether or not any priority level has exceeded its starvation threshold. Such a starvation threshold indicates that a given priority level has not been allocated a processing resource because of a higher priority level for greater than a predetermined number of such allocations. These starvation levels may be individually programmed as ratios to have effect between individual pairs of priority levels. Thus, there may be a starvation ratio for the low priority level in respect of the medium priority level and separately in respect of each of the high and high high priority levels. The starvation ratios only act up the priority hierarchy as a higher priority level will not be starved of processing resource allocation because of a lower priority level. If the determination at step 68 is that a priority level is currently exceeding its starvation threshold, then step 70 serves to issue a proceed response to the highest priority transaction request that has exceeded its starvation threshold (using a round robin selection if there are more than one of such transactions). Step 72 then increments the count value for the priority level or levels concerned so as to track the number of processing resources allocated to that priority level. If more than one counter is incremented, this step also increments the counters of the lower priority levels.
If the determination at step 68 is that a starvation threshold has not been exceeded, then step 74 selects the highest priority level with a non-zero number of queued requests. Step 76 then selects the next transaction in the selected priority level pointed to by the pointer within that queue of transaction requests. Step 78 then advances the pointer for round robin allocation within the queue and step 80 issues a proceed request to the transaction source of the selected transaction request to indicate that the transaction source should resend its transaction request this time indicating the static credit. Processing then proceeds to step 72.
If a refusal response was received at step 88, then processing proceeds to step 92 where the transaction source waits until a proceed response is received. Such a proceed response indicates that a static allocation of a processing resource has now been made for the transaction request and thus processing should proceed to step 94 where the transaction request is resent, but this time indicating that it has an associated static credit and that a processing resource has already been allocated to that transaction source in accordance with the static allocation techniques previously described. The statically allocated transaction is then completed at step 90.
Although illustrative embodiments of the invention have been described in detail herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various changes and modifications can be effected therein by one skilled in the art without departing from the scope and spirit of the invention as defined by the appended claims.
