Patent Application
20190310857
A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
Embodiments of the present invention relate generally to methods of parallel task execution and, more specifically, to efficient methods of task scheduling and work balancing in symmetric multiprocessor systems that may be coupled with processing accelerator devices and/or connected with remote computer systems.
Parallel computer systems penetrated into almost every aspect of modern technology, and parallel computing is considered by many experts as the only way of ensuring further growth of the compute-power of future computer systems and their applicability to new areas of science and technology. That brings the problem of increasing efficiency of concurrent computations, maximizing processing resource utilization and minimizing efforts spent on parallel programming to the foreground.
Unfortunately, modern parallel computing systems have certain disadvantages, which decrease the efficiency of concurrent instruction execution. Those disadvantages are typically caused by software models employed in modern parallel computer system programming. For example, there is a concept of ‘task’, known from the prior art, which implies decomposing parallel execution into portions of code to be executed concurrently and portions of data said code can operate on (collectively known as tasks). While the task concept enabled the progress in parallel programming and especially in parallel work balancing, it is not without a flaw.
Thus, to initiate concurrent execution of instructions, a set of tasks has to be generated from those instructions and from data their execution may depend on. Then, said tasks have to be queued, that is, the system needs to form one or more lists from which software threads will be picking up tasks for execution. The process of task queuing consumes time (to divide data between tasks and allocate memory for task descriptors) and memory resources (to hold task descriptors and maintain linked lists and queues): the more tasks the higher execution time and memory penalties, which negatively affects the overall performance of parallel computations.
Solving the problem of parallel work load balancing to maximize the utilization of processing resources also comes at a cost: software threads which complete execution of tasks and, consequently, whose task queues become empty have to access other threads' task queues to retrieve more tasks for execution and re-divide data between new tasks—thus blocking those other software threads from executing tasks from their tasks queues—which also negatively affects the overall performance of parallel computations.
Many modern computer systems (ranging from super-computers to mobile devices) are equipped with processing accelerator devices, which are used to increase the efficiency of parallel computations by taking part of the work load off the central processors (known in the art as heterogeneous computer systems). Typically, such systems are programmed using dedicated, highly specialized programming languages and environments, which in many cases cannot be efficiently applied to programming central processors, thus further increasing the complexity of parallel programming and inheriting all of the aforementioned limitations with regard to parallel work balancing.
Therefore, a need exists for the capability to accelerate concurrent execution of instructions, increase utilization of processing resources, minimize memory resource utilization, decrease parallel work balancing overhead, and simplify programming of heterogeneous computer systems.
The features and advantages of the present invention will become apparent from the following detailed description of the present invention in which:
An embodiment of the present invention is a method that provides for efficient execution of instructions concurrently by multiple software or hardware threads and for maximizing utilization of processing resources in heterogeneous computer systems.
Reference in the specification to “one embodiment” or “an embodiment” of the present invention means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrase “in one embodiment” appearing in various places throughout the specification are not necessarily all referring to the same embodiment.
The following definitions may be useful for understanding embodiments of the present invention described herein.
Software thread is a stream of instructions to be executed independently by one or more processors, associated with software execution context, which may comprise values of processor registers and memory locations according to particular hardware logic designs and software execution conventions.
Hardware thread denotes a processor that maintains an independent software execution context and is visible as an independent processor to the operating system and system software.
Processing device comprises one or more processors, optionally coupled with memory, which are connected with central processors within the same computer system and, as a general rule, are not visible as independent processors to the operating system. Processing devices, in order to execute instructions, typically have to be programmed by software threads that are executed by central processors.
Remote computer systems are computer systems comprising any combination of central processors, processing devices and memory storage and connected with other such systems by means of a communication device, e.g., a network interface card.
Stack memory denotes a memory region containing procedure linkage information and other data to be accessed exclusively by one software thread in accordance with software execution conventions.
Stack frame is a region of stack memory that stores data specific to a single procedure or function being executed by a software thread in accordance with software execution conventions.
According to the figure, a system embodying the present invention may comprise processors 10 (referred to in the figure as CPU, or central processing units), which may be coupled with memory 12, processing device 14, and communication device 16. Processors 10 may execute software threads, at least one software thread per processor. Said software threads may comprise instructions stored in a code region of memory 12. All software threads may have access to data stored in a data region of memory 12, and each software thread may exclusively access data stored in a stack region of memory 12.
In one embodiment of the present invention processing device 14 may comprise multiple execution units (EU) and memory containing instructions and data to be accessed by the execution units. While in other embodiments of the present invention processing device 14 may have direct access to instructions and data stored in memory 12.
One skilled in the art will recognize that systems embodying the present invention may comprise any combination of central processors and processing devices, of different hardware architecture and capabilities. Furthermore, one skilled in the art will recognize the option of employing dedicated software threads executed by processor 10 to program processing devices 14 with instructions to execute, and to transfer input and output data to and from said devices; which process will be herein referred to as executing a software thread on behalf of another processing device or system.
In an embodiment of the present invention, communication device 16 may be employed for transferring instructions and data to remote computer systems for execution and retrieving results.
According to the figure, a software thread, during its normal course of execution, may encounter a parallel execution construct or otherwise receive a request for parallel execution at block 200. An exemplary parallel execution construct is provided in block 202 and may comprise instructions to execute (instruction block in braces “{ }” in the example of this figure), data said instructions may depend on (arrays “a”, “b”, and “c” in the example if this figure), and an indication (explicit or implicit, direct or indirect, which may include, for example, providing a shared memory variable or a function argument) of the level of parallelism, i.e., the number of times the instruction block can be executed (in the example of block 202 the parallelism level equals the size of array “a”). The indication of the level of parallelism will be referred to throughout the Specification as ‘parallelism level indicator’ or ‘number of times an instruction block is to be executed’. It should be noted here that the problem of generating the actual code capable of correct concurrent execution lies beyond the scope of the present invention, while the present invention provides for means of efficient execution of such code.
Then, the software thread may form a parallel execution context at block 204. The parallel execution context may comprise, as illustrated in block 206, an instruction pointer, i.e., an address of the beginning of an instruction block to be executed concurrently; an address of the beginning of a current stack frame (referred to as Stack Frame Start); an address of the end of the current stack frame (referred to as Stack Frame End); current contents of processor registers; and a parallelism level indicator (initialized to the size of array “a” according to the example of block 202). In one embodiment of the present invention forming the parallel execution context may comprise allocating the memory structure of block 206 on the software thread's stack memory.
One skilled in the art will recognize the option of employing different means of forming the parallel execution context according to different software execution conventions. Furthermore, one skilled in the art will recognize that in other embodiments of the present invention the parallel execution context may comprise different fields to locate thread-specific data, and different processor registers as may be appropriate for certain software execution conventions or hardware architectures.
Then, at block 208, the software thread may signal to other threads that a parallel execution context is available. In one embodiment of the present invention signaling to other threads may comprise using semaphores or other conventional means of software thread synchronization provided by an operating system; while other embodiments may employ inter-processor interrupts or other means of hardware thread signaling as may be provided by a computer system.
Finally, at block 210, the software thread may optionally execute instructions from the parallel execution context (as will be further described in the example of
According to the figure, software threads may wait for a signal at block 300 to begin concurrent execution. Then, upon reception of the signal, at block 302, each software thread may retrieve parallel execution context 304. Then, at block 306, each thread may decrease the parallelism level indicator of the parallel execution context by N, wherein N corresponds to the number of times the software thread (or the processing device on whose behalf the thread operates) is capable of executing instructions of the parallel execution context. For example, software threads operating on behalf of central processors may typically decrement the parallelism level indicator by one, as traditional CPU architectures provide for execution of only one software thread per hardware thread at a time. While software threads operating on behalf of other processing devices may decrement the parallelism level indicator at least by the number of execution units available in said devices.
Further, at block 308, each software thread may allocate a local stack frame to contain thread-specific data. The size of the local stack frame may be determined as a difference between Stack Frame End and Stack Frame Start fields of the parallel execution context. At block 310, each software thread may copy the contents of the original stack frame referenced in the parallel execution context (Stack Frame Start) to the newly allocated local stack frame. Then, at block 312, each software thread may copy original register contents (referenced as Ro in the example of the figure) from the parallel execution context to local processor registers (referenced as RI in the example of the figure). After copying each register value, a check may be performed if the original register value lies within the borders of the original stack frame, and if the above condition is true, the local register value may be increased by a difference between addresses of the beginnings of the local stack frame (allocated at block 310) and the original stack frame—thus effectively enabling local processor register values to contain addresses within the local stack frame. For example, if an original register contains an address of a variable located in the original stack frame, the local register will contain the address of that variable's copy in the local stack frame.
Finally, each software thread may proceed with executing instructions referenced by the instruction pointer of the parallel execution context at block 314, and, upon completion, perform a check at block 316 of whether the parallelism level indicator becomes zero. In case the parallelism level indicator is zero, the control may be returned to block 300; otherwise, the control may return to block 306 and concurrent execution of instructions from the current parallel execution context may continue.
Different embodiments of the present invention may utilize different processor registers in the parallel execution context, as may be appropriate to comply with a particular hardware architecture and software execution convention which may be in effect in a system embodying the present invention. Similarly, if a hardware architecture or software execution convention so dictates, an embodiment of the present invention may employ any reference to data to be accessed exclusively by a software thread, other than the stack memory.
Embodiments of the present invention provide for dynamic work load balancing between multiple processors, in a manner that is adaptive to the actual work complexity and performance of a particular processor, by ensuring that a software thread that completes operations at block 314 faster than other software threads may proceed to block 306 faster and retrieve more work for execution. Persons skilled in the art will recognize that dynamic work load balancing may be performed without introducing additional memory structures for maintaining task queues and extra operations for task queue re-balancing, which further differentiates the present invention from the prior art.
For C and C++ language examples of embodiments of the present invention refer to Appendices A and B, wherein Appendix A comprises a program that illustrates the use of concurrent operations for computing a set of Fibonacci numbers; and Appendix B furnishes an exemplary implementation of a run-time library that enables code execution by multiple concurrent threads and automatic parallel work balancing. The provided code excerpts do not constitute a complete concurrent instruction execution system, are provided for illustrative purposes, and should not be viewed as a reference implementation with regard to both their functionality and efficiency.
One skilled in the art will recognize the option of employing different data types and thread interaction schemes as may be appropriate for a given operating system or programming environment while still remaining within the spirit and scope of the present invention. Furthermore, one skilled in the art will recognize that embodiments of the present invention may be implemented in other ways and using other programming languages.
The techniques described herein are not limited to any particular hardware or software configuration; they may find applicability in any computing or processing environment. The techniques may be implemented in logic embodied in hardware components. The techniques may be implemented in programs executing on programmable machines such as mobile or stationary computers, personal digital assistants, set top boxes, cellular telephones and pagers, and other electronic devices, that each include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and one or more output devices. Program code is applied to the data entered using the input device to perform the functions described and to generate output information. The output information may be applied to one or more output devices. One of ordinary skill in the art may appreciate that the invention can be practiced with various computer system configurations, including multiprocessor systems, minicomputers, mainframe computers, and the like. The invention can also be practiced in distributed computing environments where tasks may be performed by remote processing devices that are linked through a communications network.
Each program may be implemented in a high level procedural or object oriented programming language to communicate with a processing system. However, programs may be implemented in assembly or machine language, if desired. In any case, the language may be compiled or interpreted.
Program instructions may be used to cause a general-purpose or special-purpose processing system that is programmed with the instructions to perform the operations described herein. Alternatively, the operations may be performed by specific hardware components that contain hardwired logic for performing the operations, or by any combination of programmed computer components and custom hardware components. The methods described herein may be provided as a computer program product that may include a machine readable medium having stored thereon instructions that may be used to program a processing system or other electronic device to perform the methods. The term “machine readable medium” used herein shall include any non-transitory medium that is capable of storing or encoding a sequence of instructions for execution by the machine and that cause the machine to perform any one of the methods described herein. The term “machine readable medium” shall accordingly include, but not be limited to, solid-state memories, optical and magnetic disks. Furthermore, it is common in the art to speak of software, in one form or another (e.g., program, procedure, process, application, module, and so on) as taking an action or causing a result. Such expressions are merely a shorthand way of stating the execution of the software by a processing system to cause the processor to perform an action or produce a result.
While this invention has been described with reference to illustrative embodiments, this description is not intended to be construed in a limiting sense. Various modifications of the illustrative embodiments, as well as other embodiments of the invention, which are apparent to persons skilled in the art to which the invention pertains are deemed to lie within the spirit and scope of the invention.
This is a continuation of application Ser. No. 13/867,803, filed on Apr. 22, 2013.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13867803 | Apr 2013 | US |
| Child | 16433997 | US |
