1. Field of the Invention
The present invention relates generally to increasing utilization and overall performance in multi-threading microprocessors. More particularly, the present invention relates to maximizing the efficiency of processors by concealing the switch latency of a multi-threading microprocessor by switching the processor to a different software thread when a mispredicted branch has occurred.
2. Description of the Related Art
In a conventional computer system, microprocessors are typically required to run more than one program (which may include more than one software thread). The computer system utilizes an operating system (OS) to direct the microprocessor to run each of the programs based on priority. The simplest type of priority system simply directs the OS to run the programs in sequence (i.e., the last program to be run has the lowest priority). In other systems, the priority of a program may be assigned based on other factors, such as the importance of the program, how efficient it is to run the program, or both. Through priority, the OS is then able to determine the order in which a program or a software thread is executed.
One way of optimizing the performance of a computer system is to design it so that its microprocessor(s) are being utilized as much as possible. Unfortunately, one of the traditional constraints of a processor was that if a program or a software thread being executed was unable to continue and stalled by an event (e.g., because the event requires a long latency memory access), the processor would experience idle cycles for the duration of the stalling event, thereby decreasing the overall system performance.
Recent developments in processor design have allowed for multi-threading, where two or more distinct threads are able to make use of available processor resources. One particular form of multi-threading is Switch on Event Multi-Threading (SoEMT). In SoEMT, if one thread is stalled by an event, the processor (not the OS) may switch the execution context to a second thread. Such an event is known as a switching event. The second thread then takes control over the processor and executes its program until the program is finished or another switching event occurs, upon which the processor may switch back to execute the original thread or execute a different thread.
While the ability to switch the processor between threads can dramatically increase processor utilization, the overall performance of a SoEMT system may still be hampered by the fact that switching from one software thread to the other takes a predetermined amount of time. This switch latency includes the overhead to detect and process the thread switch as well as flushing the execution pipeline and refilling it with the new thread's instructions. When processors encounter a large number of switching events, a significant amount of processor time may be devoted to switch latency, which would diminish the performance advantage gained from switching.
One way of eliminating switch latency is to utilize a processor design known as simultaneous multi-threading (SMT), which allows multiple threads to issue instructions each cycle. Unlike SoEMT, in which only a single thread is active on a given cycle, SMT permits all threads to compete for and share processor resources at the same time. Unfortunately, SMT systems do not represent an entirely adequate method of eliminating switch latency because SMT systems are extremely expensive to manufacture. SMT systems are also very complex and require far more computing power to operate than a SoEMT system. Therefore, is it desirable to have a method and apparatus that reduces or eliminates switch latency in a multi-treading system without incurring the cost of manufacturing a SMT system.
The present invention will be readily understood by the following detailed description in conjunction with the accompanying drawings. To facilitate this description, like reference numerals designate like structural elements.
A method and apparatus for concealing switch latency in a multi-threading computer system is provided. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be understood, however, to one skilled in the art, that the present invention may be practiced without some or all of these specific details. In other instances, well known process operations have not been described in detail in order not to unnecessarily obscure the present invention.
In SoEMT system 10, front end module 18 receives and decodes an input containing instructions while performing the necessary diagnostics. The instructions are eventually transmitted to state module 12 and execution module 20. At the same time, execution module 20 carries out the preceding instructions and generates an output. By decoding one instruction while the preceding instruction is executing, the microprocessor saves time. This “assembly line” is called pipelining. As is well known in the art, the pipeline may have many steps to accommodate numerous sequential instructions at the same time, one or more at each stage in the pipeline (current microprocessors can accommodate up to six instructions per stage).
SoEMT system 10 differs from single threading systems by duplicating IPs and register files within state module 12 while using only one true processor. IPs 14a-b and register files 16a-b are monitored by switch logic module 22, which determines which software threads are executed. While SoEMT system 10 is configured to handle only two threads in
To avoid this break in the flow of instructions, microprocessors attempt to predict what the branch instruction will do, based on a record of what the particular branch instruction did before. The microprocessor then decides which instruction to load next into the pipeline based on the branch prediction. This speculative execution has the potential to save lot of processor time, but only if the branch prediction is correct. If the prediction turns out to be wrong, it is termed a mispredicted branch. The processor must then “pay” an unavoidable time penalty by flushing the pipeline to discard all of the calculations that were based on the mispredicted branch.
Referring back to
Assuming that load command 30 is a switching event that can be rescheduled, switch logic module 22 will reschedule the switch from software thread 24 to software thread 26 until it detects a mispredicted branch. After mispredicted branch 28 is detected, the processor is forced to pay the mispredicted branch penalty by flushing the pipeline and discarding all the calculations based on the incorrect prediction. However, at the same time, switch logic module 22 switches the processor from executing software thread 24 to executing software thread 26.
Because the switch has been rescheduled to occur at the same time as the branch misprediction penalty, the switch latency is at least partially, if not fully concealed by the branch misprediction penalty. Normally, switch latency is unavoidable because it takes perhaps about 15 to about 20 clocks to switch the processor to a different software thread. However, because the execution penalty resulting from a branch misprediction is also unavoidable, it is advantageous to schedule switching at the same time, thereby consolidating the latencies as much as possible.
Through this process, the switch latency is effectively reduced or eliminated by the length of time it takes to flush the pipeline due to the incorrect branch prediction. Since today's processors tend to have longer and longer pipelines compared to older processors, the reduction in switch latency may be substantial especially since mispredicted branches typically occur more frequently than switching between software threads. This is particularly true in applications that have a lot of switching events that threaten to diminish the performance advantage gained by multi-threading.
If an immediate switching event is detected, then the processor switches threads in a block 48. If an immediate switch is not required, then method 42 proceeds to a block 50, which determines whether or not a mispredicted branch is outstanding. Examples of switching events that can be rescheduled are switches necessary because of switches resulting from cache misses, and time quanta switches (which are discussed in greater detail below).
At the same time switching events are being detected in block 44, method 42 may also begin at a block 52 when a mispredicted branch is detected. After detection, a mispredicted branch indicator is set in a block 54. The indicator allows block 50 to determine whether or not a mispredicted branch is outstanding. If a switch event has been detected and a mispredicted branch is outstanding, then the switch latency can be concealed by the mispredicted branch latency, and method 42 proceeds to block 52 where the processor switches threads. If a mispredicted branch has not been detected in block 52, then the mispredicted branch indicator from block 54 has a reset value. Therefore, block 50 determines that a mispredicted branch is not outstanding and method 42 proceeds to blocks 56 and 58 at the same time.
In block 56, an outstanding switch request indicator is set because a switch event was previously detected in block 44. Then a time-out counter is started in block 58 to ensure that the switch will occur regardless of whether a mispredicted branch is detected in block 52. (Note that another time-out counter may be used to trigger a switch in the absence of any other switching event). Therefore, a block 60 determines whether the time quantum has passed or whether a switch request and a mispredicted branch are both outstanding. If either of these conditions is met, then method 42 proceeds to block 50 where the processor switches software threads. At the same time, the mispredicted branch indicator and the time-out counter are reset.
The amount of time that the time-out counter is set to is known as the time quantum. The time quantum may vary from application to application, however it is always set to an amount of time that ensures fairness between the software threads. In other words, even though method 42 is waiting for a mispredicted branch to occur in block 60 and therefore an opportunity to conceal the switch latency, after the time quantum has passed, the processor must be switched so that other threads are not ignored for too long. Typically, the time quantum is less than about 1,000 clocks and preferably about 200 clocks.
In summary, the present invention provides for a method and apparatus for concealing switch latency in a multi-threading computer system. The present invention includes a processor having a switch logic module, which detects switching events and mispredicted branches in a software thread. When a switching event is detected, the switch logic module determines whether or not an immediate switch to a different software thread is required. If an immediate switch is not required, then the switch logic module determines whether a mispredicted branch is outstanding. If a mispredicted branch is outstanding, then the processor switches software threads, concealing at least part if not all of the switch latency in the unavoidable mispredicted branch latency. If a mispredicted branch is not detected, then the switch logic module delays the switch for a certain time quantum, and then executes the switch in the interest of fairness.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention. Furthermore, certain terminology has been used for the purposes of descriptive clarity, and not to limit the present invention. The embodiments and preferred features described above should be considered exemplary, with the invention being defined by the appended claims.
The present application is a continuation of U.S. patent application Ser. No. 09/753,766, filed Dec. 29, 2000.
Number | Date | Country | |
---|---|---|---|
Parent | 09753766 | Dec 2000 | US |
Child | 11388321 | Mar 2006 | US |