The present disclosure relates generally to microprocessor systems, and more specifically to microprocessor systems capable of multi-threaded operation.
Many modern computer systems are able to execute more than one distinct software program, or “thread”, without having to explicitly save the state for one thread and restore the state for another thread. For this reason they are referred to as “multi-threaded” computer systems. In one older approach, called sequential multi-threaded operation, the operating system or other control mechanism permits the several threads to share resources by permitting each thread that is a candidate for execution to operate in sequence on the processor. Changing between threads may be referred to as thread switching. In some of these older approaches, threads are switched when the currently executing thread executes for a certain period or reaches a point when it cannot proceed, such as waiting for a memory access or an I/O transfer to finish. The selection of the next thread to be switched in (permitted use of execution resources) may be made on the basis of strict priority. In other approaches, a round-robin approach may be used in thread switching.
Several modern microprocessors support what is known as simultaneous (or concurrent) multi-threaded operation. In these processors, several threads may execute simultaneously sharing resources in parallel rather than in sequence. However, in many cases there will still be more threads that need to be executed than there are hardware execution resources for thread execution. Therefore there will still be a need for thread switching, at least with respect to some resources, when using these processors.
One problem that may arise when thread switching is deadlock. Deadlock may be defined as the condition when a set of processes using shared resources or communicating with each other are permanently blocked.
A similar problem is livelock. Livelock may be defined as the condition when attempts by two or more processes to acquire a resource run indefinitely without any process succeeding.
Thus deadlock and livelock are two examples of potential problems that may arise when switching threads in a multi-threaded system. Improved algorithms for controlling switching threads and algorithms that address deadlock or livelock concerns may continue to be desirable for their ability to improve overall multi-threaded execution.
The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:
The following description describes techniques for permitting the execution of multiple threads in a microprocessor system. In the following description, numerous specific details such as logic implementations, software module allocation, bus signaling techniques, and details of operation are set forth in order to provide a more thorough understanding of the present invention. It will be appreciated, however, by one skilled in the art that the invention may be practiced without such specific details. In other instances, control structures, gate level circuits and full software instruction sequences have not been shown in detail in order not to obscure the invention. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation. The invention is disclosed in the form of hardware within a microprocessor system. However, the invention may be practiced in other forms of processor such as a digital signal processor, a minicomputer, or a mainframe computer. The invention may also be practiced within the software of an operating system that may execute on any form of digital processor.
In one embodiment, the determination of which threads to switch and when to switch them is supported by a thread prioritizer and a switch enabler. In one embodiment, the thread prioritizer provides relative thread fairness indicators for each pair of threads that are enabled to execute in the processor. A separate fairness counter may be used to support each pair of threads, configured to count up when one thread is executing and to count down when the other thread is executing. A relative fairness indication may be derived from the value of the counter. Once the relative fairness is known for each pair of ready threads, an overall absolute fairness ranking may be determined. In one embodiment, the switch enabler may include a forward progress counter for each thread currently executing. In this embodiment, the value of the forward progress counter may be used to determine whether to permit a voluntary switching out of that particular thread.
Referring now to
The processors CPU A 110, CPU B 120, and CPU C 130 may be connected via a system bus 112 with each other and with chipset 140. The connection via the system bus 112 and the chipset 140 permits the processors to access system memory 150 and various input/output (I/O) devices, such as a controller for graphics 160 and various program storage devices. In other embodiments, the processors may be connected via some other form of interconnect such as point to point interconnections. Such program storage devices may include a system fixed disk 170 and a drive for removable media 180. In various embodiments, drive for removable media 180 may be magnetic tape, removable magnetic disks, diskettes, electro-optical disks, or optical disks such as CD-ROM or DVD-ROM. The I/O devices may be connected to the chipset 140 via a dedicated interface, such as advanced graphics port (AGP) 144, or via general-purpose interfaces such as peripheral component interconnect (PCI) bus 152, universal serial bus (USB) (not shown), or integrated drive electronics (IDE) bus (not shown). Additional I/O devices may include connectivity to a local area network (LAN) 146 or a wide area network (WAN) 148. In other embodiments, many other interfaces may be used.
In one embodiment, an operating system may be installed on system fixed disk 170 and the kernel of the operating system may be loaded into system memory 150. In other embodiments, the operating system may be loaded or execute over the LAN 146 or WAN 148. The operating system may control the thread switching in the processors CPU A 110, CPU B 120, and CPU C 130. In other embodiments, the hardware or firmware within the processors may control the thread switching, or the combination of the hardware or firmware within the processors together with an operating system may control the thread switching.
Referring now to
The instructions within level 1 (L1) cache 210 may include those ready for execution or those recently executed. An instruction prefetch/fetch 220 circuit attempts to load the instructions for the various threads being executed from L1 cache 210. If the requested instructions are not in L1 cache 210, circuitry seeks out and loads the instructions from higher levels of cache or even system memory (not shown). Instruction prefetch/fetch 220 is assisted by a thread control logic 270 that informs instruction prefetch/fetch 220 via prefetch/fetch/issue control signal 280 which threads should be switched in for execution, and therefore which particular instructions should be fetched. The quality of the prefetching may be enhanced by thread fairness and forward progress information.
After the instructions for the several threads are fetched, they are temporarily stored in buffers/issue logic 230. The buffers/issue logic 230, also receiving the issue control signal 280, also disperses the instructions to the following block, register rename/instruction decode/register read 240 circuitry. As part of the dispersement, buffers/issue logic may report one or more issue counts signals 282 back to the thread control logic 270. Immediately prior to their execution, the register environment needed for the instructions is setup by register rename/instruction decode/register read 240 circuitry. Then the instructions are executed by a group of several execution units 250. After execution, exceptions are handled and retirement is made by exception/retirement 260 circuitry. Exception/retirement circuitry 260 may also return one or more retirement counts signals 284 back to the thread control logic 270.
Referring now to
In order to determine these two indicators, thread prioritizer 310 may utilize a number of fairness counters. In one embodiment, there is a fairness counter allocated for each pair of prospective threads. The number of prospective threads and other system information may be provided the thread prioritizer 310 by the configuration 350 signal. If there are N prospective threads, then there may be N(N−1)/2 fairness counters allocated. In one embodiment the fairness counters may be signed counters capable of being incremented or decremented. In other embodiments, a pair of unsigned counters may be used. In another embodiment, a single unsigned register or counter may be shared between two threads with the count corresponding to the degree of bias for the currently biased thread. In additional embodiments, the fairness counters may be constructed in software of an operating system. The fairness counter of a given pair of threads may be incremented or decremented by any of a number of execution “events.” These events may include processor cycles, weighted processor cycles, instruction counts, weighted instruction counts, or any other repetitive event correlated to execution. In some embodiments, the weighting may vary in time, in response to operator type or in response to resource use. Complicated operations may be weighted more heavily than simpler ones. Execution information 352 signals may include any or all of the above events, and in addition other information such as the number of instruction completed or the number of cycles executed. In
The
Certain switch stimuli may not be postponed, such as responding to a non-maskable interrupt or other non-maskable stimuli. Such stimuli may be referred to as mandatory stimuli. Such stimuli may bypass or ignore any determination of forward progress within switch enabler 320. In contrast, certain switch stimuli may be postponed at the system's discretion. These maskable stimuli may be referred to as volative or voluntary stimuli, because thread switching in response to them may be considered a voluntary act. In order to exercise this discretion, switch enabler 320 may include a series of forward progress counters 322, 324.
In one embodiment, there is a forward progress counter allocated for each presently executing thread. Information about which threads are currently executing may be presented in the execution information 352 signal. Information about system capabilities such as number of threads that may be concurrently executed may be presented in the configuration 350 signal. If there may be M actually executing threads, then there may be M forward progress counters allocated. In one embodiment the forward progress counters may be unsigned counters capable of being incremented or reset. In other embodiments the forward progress counters may be software structures in an operating system. The forward progress counter of a given presently executing thread may be incremented by any of a number of execution “events” of that thread. As in the case of the fairness counters discussed above, these events may include processor cycles, weighted processor cycles, instruction counts, weighted instruction counts, or any other repetitive event correlated to execution. In the
The absolute thread priority 354 signal and thread fairness 356 signal from thread prioritizer 310 and the thread switch enables 358 signals from the switch enabler 320 may be presented to a next thread selector 330. Next thread selector 330 may combine these signals, in conjunction with execution information 352 and configuration 350, to provide a circuit such as thread control logic 270 of
Referring now to
When the content of the fairness counter is either below the A threshold 412 or above the B threshold 416, for example at count 422 or count 428, respectively, then it can be said that the relative execution history is now unfairly biased. At count 422 it can be said that the relative execution history is “unfairly biased towards A” whereas at count 428 it can be said that the relative execution history is “unfairly biased towards B”. A determination of unfair bias towards one thread of a pair may be used to determine the absolute ordering or as a portion of switching stimuli 370 to switch to the other thread. The actual count, the content of the fairness counter and the threshold used, may additionally be used to indicate quantitatively the unfairness of the bias.
If the count was unrestricted, when one thread has a natural pause in execution, such as when waiting for data from an I/O device, the other thread could continue incrementing or decrementing the count of the fairness counter. (Note, however, that in some embodiments there may be execution events even when a thread is not running.) This may result in the paused thread being given too much access to future allocation of resources and may lead to temporal starvation for the previously running thread. (Here there may be defined two forms of starvation, a condition when a process is indefinitely denied access to a resource while other processes are granted access to the resource. Absolute starvation may be when the thread never gets to run, whereas temporal starvation may be when the thread is prevented from running for extended periods of time.) Therefore, there are two limits placed upon the fairness counter, an A limit 410 and a B limit 418. When execution events for thread A are continuing with no corresponding execution events for thread B, the count will eventually reach A limit 410. When this happens, the fairness counter stops decrementing in response to additional execution events for thread A. When count 420 is below the A limit 410, the fairness counter may not decrement until such time as offsetting thread B events increment the fairness counter above the A limit 410. Similarly the fairness counter may not increment above the B limit 418 until offsetting thread A events decrement the fairness counter below the B limit 418.
The behavior of the fairness counter may be summarized in Table I below. Note that in Table I certain inequalities may be expressed in terms of or rather than strict inequalities to ensure that a determination is made. In other embodiments, the location of the non-strict inequalities may be changed.
With more than a single pair of threads, as in the
Referring now to
A switch enabler 320, including several forward progress counters 322, 324, may then qualify portions of switch stimuli 370 signals. The contents of a given forward progress counter may not affect a mandatory stimulus to remove a currently executing thread from execution. However, a voluntary stimulus to remove a currently executing thread from execution may be disabled until the count of the corresponding forward progress counter is greater than or equal to the threshold 522. In other embodiments, the voluntary stimulus may be disabled until the count of the corresponding forward progress counter is strictly greater than the threshold 522. When the count is at count 510, the voluntary stimulus may be disabled. When the count is at count 512, the voluntary stimulus may be enabled. In either case the qualified switch stimuli are then posted to the next thread selector 330 as thread-switch enables 358. The individual forward progress counters may post either the actual count, perhaps with relevant threshold information, to logic within switch enabler 320, or just the indication of exceeding the threshold. In other embodiments quantities derived from the count, such as ratios, may be used. In any case the forward progress counter posts an executing thread voluntary switch enabled indicator.
Referring now to
Referring now to
The
If the event is determined in decision block 712 to be not a thread A event, then the process enters decision block 740 where it is determined whether the event is a thread B event. If not, then the process re-enters block 710. But if so, then in decision block 714 it is determined whether the fairness count exceeds the B limit. If so, then in block 730 an unfair bias to thread B indicator is posted. If not, then in block 716 the fairness counter is incremented, and the process moves on to decision block 718.
In decision block 718 it is determined whether the fairness count is less than the A threshold. If so, then an unfair bias to thread A indicator is posted. If not, then the process continues to decision block 720. In decision block 720 it is determined whether the fairness count is less than the zero reference point. If so, then a fair bias to thread A indicator is posted at block 736. If not, then the process continues to decision block 722. In decision block 722, it is determined whether the fairness count is less than the B threshold. If so, then a fair bias to thread B indicator is posted at block 738. If not, an unfair bias to thread B indicator is posted at block 730.
Referring now to
When a thread begins execution, in block 810 the forward progress counter is initialized, which may include resetting the forward progress counter to zero, or some other bias point, and loading the thresholds to be used. Then in block 812 the process 800 waits for a forward progress event, which may be an execution event associated with the currently executing thread. When an event is detected, in block 814 the forward progress counter is incremented, and the process 800 returns to block 812.
At the same time the process 800 enters the block 812, it also enters block 820 wherein it waits for a thread switch stimulus event. When a thread switch stimulus event occurs, then in decision block 822 it is determined whether the thread switch stimulus event is of type 1. If so, then in block 830 it is determined whether the forward progress count is greater than or equal to the A threshold for stimulus type 1. If so, then a signal is sent conveying the information that a type 1 switch stimulus is enabled, in block 840. If not, then in block 836 a signal is sent conveying the information that no voluntary switch stimulus is enabled. In one embodiment, a signal is sent only if there has been a change in status from the time when the preceding signal was sent.
If, in decision block 822, it was determined that the thread switch stimulus event is not of type 1, then in decision block 824 it is determined whether the thread switch stimulus event is of type 2. If so, then in decision block 832 it is determined whether the forward progress count is greater than or equal to the A threshold for stimulus type 2. If so, then a signal is sent conveying the information that a type 2 switch stimulus is enabled, in block 842. If not, then in block 836 a signal is sent conveying the information that no voluntary switch stimulus is enabled.
If, in decision block 824, it was determined that the thread switch stimulus event is not of type 2, then in decision block 826 it is determined whether the thread switch stimulus event is of type 3. If so, then in decision block 834 it is determined whether the forward progress count is greater than or equal to the A threshold for stimulus type 3. If so, then a signal is sent conveying the information that a type 3 switch stimulus is enabled, in block 844. If not, then in block 836 a signal is sent conveying the information that no voluntary switch stimulus is enabled.
The individual bias indicators for each pair of threads posted in the
In the foregoing specification, the invention has been described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
Number | Name | Date | Kind |
---|---|---|---|
3771138 | Celtruda et al. | Nov 1973 | A |
5357617 | Davis et al. | Oct 1994 | A |
5386561 | Huynh et al. | Jan 1995 | A |
5404469 | Chung et al. | Apr 1995 | A |
5430850 | Papadopoulos et al. | Jul 1995 | A |
5499349 | Nikhil et al. | Mar 1996 | A |
5524263 | Griffth et al. | Jun 1996 | A |
5791522 | Lee et al. | Aug 1998 | A |
5809271 | Colwell et al. | Sep 1998 | A |
5809522 | Novak et al. | Sep 1998 | A |
5892959 | Fung | Apr 1999 | A |
5900025 | Sollars | May 1999 | A |
5968160 | Saito et al. | Oct 1999 | A |
5968167 | Whittaker et al. | Oct 1999 | A |
5996085 | Cheong et al. | Nov 1999 | A |
5999932 | Paul | Dec 1999 | A |
6018759 | Doing et al. | Jan 2000 | A |
6052708 | Flynn et al. | Apr 2000 | A |
6052709 | Paul | Apr 2000 | A |
6076157 | Borkenhagen et al. | Jun 2000 | A |
6088788 | Borkenhagen et al. | Jul 2000 | A |
6092175 | Levy et al. | Jul 2000 | A |
6105051 | Borkenhagen et al. | Aug 2000 | A |
6105127 | Kimura et al. | Aug 2000 | A |
6115709 | Gilmour et al. | Sep 2000 | A |
6212544 | Borkenhagen et al. | Apr 2001 | B1 |
6233599 | Nation et al. | May 2001 | B1 |
6256775 | Flynn | Jul 2001 | B1 |
6389449 | Nemirovsky et al. | May 2002 | B1 |
6430593 | Lindsley | Aug 2002 | B1 |
6542921 | Sager | Apr 2003 | B1 |
6567839 | Borkenhagen et al. | May 2003 | B1 |
6675190 | Schabernack et al. | Jan 2004 | B1 |
6697935 | Borkenhagen et al. | Feb 2004 | B1 |
6757811 | Mukherjee | Jun 2004 | B1 |
6757897 | Shi et al. | Jun 2004 | B1 |
6792525 | Mukherjee et al. | Sep 2004 | B2 |
6928647 | Sager | Aug 2005 | B2 |
7065762 | Duda et al. | Jun 2006 | B1 |
20020087844 | Walterscheidt et al. | Jul 2002 | A1 |
20030154235 | Sager | Aug 2003 | A1 |
20030158885 | Sager | Aug 2003 | A1 |
Number | Date | Country |
---|---|---|
0346003 | Dec 1989 | EP |
0352935 | Jan 1990 | EP |
0725335 | Aug 1996 | EP |
0747816 | Dec 1996 | EP |
0 768 608 | Apr 1997 | EP |
0768608 | Apr 1997 | EP |
0827071 | Mar 1998 | EP |
0 856 797 | Aug 1998 | EP |
0768608 | Aug 1998 | EP |
0962856 | Dec 1999 | EP |
2 311 880 | Oct 1997 | GB |
WO 9921082 | Apr 1999 | WO |
WO 9921088 | Apr 1999 | WO |
WO 9921089 | Apr 1999 | WO |
WO 0177820 | Oct 2001 | WO |
WO 0177820 | Oct 2001 | WO |
Number | Date | Country | |
---|---|---|---|
20030233394 A1 | Dec 2003 | US |