Symmetric multiprocessing (SMP) is a computer architecture that provides fast performance by making multiple CPUs available to complete individual processes simultaneously (multiprocessing). Any idle processor can be assigned any task, and additional CPUs can be added to improve performance and handle increased loads. A chip multiprocessor (CMP) includes multiple processor cores on a single chip, which allows more than one thread to be active at a time on the chip. A CMP is SMP implemented on a single integrated circuit. Thread-level parallelism (TLP) is the parallelism inherent in an application that runs multiple threads at once. A goal of CMP is to allow greater utilization of TLP.
Parallel programming languages (e.g., OpenMP, TBB, CILK) are used for writing multithreaded applications. The tasks to be performed in a multithreaded application may have well defined tasks to be performed in parallel (parallelism) so that different cores can be assigned different tasks. However, the tasks to be performed may exhibit irregular parallelism (e.g., operate on tree-based dynamic structures). Workqueuing is identification of tasks that can be performed in parallel and the queuing of these tasks. The tasks queued may then be dequeued and processed by processor cores having available processing power. The workqueuing execution enables a user to exploit irregular parallelism among dynamic data structures. Workqueuing is an effective technique to achieve high scalability performance for large number of processors.
In OpenMP, the workqueuing model is supported by taskq and task pragmas. The taskq pragma specifies the environment within which the enclosed units of work (tasks) are to be executed and the task pragma specifies the unit of work (task). When a taskq pragma is encountered a master thread initializes a queue based on the taskq pragma and executes the code within a taskq block serially. When a task pragma is encountered it conceptually adds the task to the queue created by the master thread. A captureprivate clause may be used ensure that a private copy of the link pointer is captured at the time each task is being enqueued. Slave threads dequeue the tasks from the queue and execute them.
In case of data dependence existing between master and slave threads, a value (heap variable) from a master thread (value producer) may need to be passed to a worker thread (value consumer). To avoid the value being overwritten by the master thread before the previous data is actually used by a slave thread, a memory copy operation may be used for passing the value from the master to the slave. The memory copy operation copies the data from a master thread to a slave thread to ensure that master and slave threads operate on different memory location. However, if excessive memory copy operations are performed, bus bandwidth to the shared CMP/SMP memory hierarchy can become saturated. Saturated bus bandwidth may lead to the memory copying experiencing high performance penalty.
Another possible approach for passing the value between master and slave threads is by having the master thread allocate memory space for each task. The master thread saves the data to these memory locations for later use by the slave threads. The slave threads read data from associated memory locations, perform computations on the data, and then deallocate the memory space after completion of the computation. This approach requires frequent memory allocation/deallocation operations, which causes poor memory system performance in the CMP/SMP memory hierarchy.
The features and advantages of the various embodiments will become apparent from the following detailed description in which:
When slave threads in a multithreading application are dependent upon data from a master thread the data needs to be made available to the slave threads so the slave threads can use the data before the data is possibly overwritten by the master thread. The data can be made available to the slave threads by having the master thread allocate memory space for the slave threads. That is, the memory may hold data captured by the master thread for use by the slave threads. The size of the memory space may be based on the number of threads that may be performing parallel tasks. For example, if four threads may be performing parallel tasks the memory space may include eight addresses for storing data to allow for pipelined operations.
The pointers to the memory locations may be dynamically renamed in order to pass the arguments for heap variables to the slave threads. In order to carry out argument passing via pointer renaming for parallel programming languages (e.g., OpenMP, TBB, CILK) with workqueuing model, an associated compiler and library may be required to provide support. The pointer renaming may be performed automatically during execution (run-time) and be managed by the library. The run time library may be responsible for mapping virtual pointers to the physical pointers. The application of pointer renaming may be transparent to programmers.
A workqueuing parallel programming language compiler may need to be modified so that it identifies heap variables in arguments being passed from master thread to slave threads and inserts hint information recording the number of heap variables and their data type. It determines the first use and last use instructions for these variables in the task through data dependence analysis, which is easily done in the compiler. The compiler then inserts a first specific instruction before first use of heap variables and a second specific instruction after last use of heap variables. The first specific instruction marks the pointer as a logic pointer (determination of a physical pointer associated therewith is determined at runtime and is discussed with respect to the runtime library actions). The second specific instruction is to set the state of entry to null to indicate all computations on the memory data have been completed, and this physical pointer is available for writing new data into the memory pointed by it.
It should be noted that the xxx after pragma in
The compiler will identify that there is one heap variable (pImageObject) and will insert hint information regarding the fact that there is one heap variable and it's data type is CImageObject. The compiler will then determine that first and only use of the heap variable is for the extraction (statement 9). Accordingly, the compiler puts the first specific instruction before statement 9 and the second specific instruction after statement 9.
A workqueuing parallel programming language library may need to be modified to perform a series of actions at runtime. First, when encountering the taskq pragma a master thread needs to initialize a queue and create a cyclic pointer buffer based on the hint information for the number of heap variables and their data types passed from compiler. The buffer size is equal to the task queue size, so when the queue is full the buffer is also full. The buffer captures a pointer to a memory location in the queue where the data is stored and a state bit representing a current status of the entry. The states of the entry may be null, ready and reading and will be defined in more detail later. Initially, each entry is initialized with a pointer of same type as the passed argument and a null state (meaning data can be written thereto or is available for tasking). The pointer in each entry is captured by invoking the same construction function used to capture the pointer in the master thread.
When the runtime library passes a heap variable (pointer) from the master thread to a slave thread through some dedicated clause, the pointer of the master thread is exchanged with a pointer for an entry in the buffer having a null state (the pointers are dynamically renamed). The state value for the entry in the buffer is changed to ready indicating that it is ready for later use by a slave thread (is available for processing). Since the master thread had its pointer renamed to another value, in next iteration, the master thread will save data to another memory location and avoid the value overwriting to same memory area. The dedicated clause may be a captureprivate clause that is typically used to capture the pointer modified to also perform the pointer renaming thus not requiring any programming modification on the part of a user. Alternatively, the captureprivate may remain the same and a new clause (e.g., pointer-renaming) may be used to initiate the pointer renaming feature. This alternative would enable a user to option of when to initiate this feature. It should be noted that
When the runtime library encounters the first specific instruction and initiates a slave thread it will go through the cyclic pointer buffer to search for an entry with ready state (the memory data is available). After finding such an entry in the buffer, the runtime library will associate the logical pointer in the slave thread with the physical pointer for that entry and will change the state of that entry in the buffer to reading. Setting the state in the buffer to reading indicates a thread is using this pointer and thus is not available for other threads to use.
When the runtime library completes a slave thread and executes the second specific instruction, the state value of the entry containing this pointer is set to null indicating that the pointer in this entry is not in use and new data can be written to the associated memory location referenced by the pointer in this entry.
The process described above in
The cyclic pointer buffer is shared among all threads (including master and slave threads) and the frequent access to this shared central resource may cause high synchronization overhead on large number of processors, which will reduce the effectiveness of pointer renaming mechanism.
According to one embodiment, synchronization overhead associated with a centralized shared pointer buffer can be reduced by implementing a distributed pointer buffer scheme. The distributed pointer buffer scheme includes several pointer buffers being created, and only a small number of threads can access each pointer buffer. This reduces the contention to the shared resource, which in turn decreases the synchronization overhead. As an example, a 16 thread application could have four buffers created with each buffer providing access to four threads. The master thread may distribute the pointer (e.g., decoded frames in
If the workload of a distributed buffer varies dramatically, the workload is imbalanced. Workload imbalance in a multi-threaded application may be reduced by implementing a work stealing strategy. The work stealing strategy entails an idle processor taking tasks from a busy processor. If a current pointer buffer is empty, then runtime library will steal a valid pointer element from another random selected pointer buffer and execute work on this stolen data on an idle physical processor. For example, the computation of feature extraction in a video mining application (e.g.,
The task queue and the cyclic pointer buffer (e.g.,
Data locality is an important factor affecting the performance of parallel programs running on SMP and CMP systems. A good balance scheme should evenly distribute the workload among available CPUs 810 and locate the task being performed by a processor (thread) close to the data that the task will be performed on in order to reduce the communication and idle time. That is, it is beneficial to have one thread (e.g., slave thread) running on one physical CPU 810 perform computation on data produced by another thread (e.g., master thread) running on a physical CPU 810 sharing memory therewith. For example, if the first CPU in
In addition for the work stealing strategy discussed above, it is good for performance if an idle CPU 810 steals data from a pointer buffer sharing cache 840 with the idle CPU. For example, if the first and second CPU in
Since the number of processors and cache organization (e.g., number of levels, cache size, private or shared) can be determined at runtime, the runtime library can schedule the task close to the data to preserve the data locality.
Various embodiments were described above with specific reference made to the OpenMP parallel processing language. The various embodiments are in no way intended to be limited thereto but could be applied to any parallel programming language (e.g., CILK, TBB) without departing from the current scope. The compilers and libraries associated with any parallel programming language could be modified to incorporate a pointer renaming scheme for fast argument passing.
The various embodiments were described with respect to multiprocessor systems with shared memory (e.g., SMP, CMP) but are not limited thereto. The various embodiments can be applied to any system having multiple parallel threads being executed and a shared memory amongst the threads without departing from the scope. For example, the various embodiments may apply to systems that have a plurality of microengines that perform parallel processing of threads.
Although the disclosure has been illustrated by reference to specific embodiments, it will be apparent that the disclosure is not limited thereto as various changes and modifications may be made thereto without departing from the scope. Reference to “one embodiment” or “an embodiment” means that a particular feature, structure or characteristic described therein is included in at least one embodiment. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment” appearing in various places throughout the specification are not necessarily all referring to the same embodiment.
An embodiment may be implemented by hardware, software, firmware, microcode, or any combination thereof. When implemented in software, firmware, or microcode, the elements of an embodiment are the program code or code segments to perform the necessary tasks. The code may be the actual code that carries out the operations, or code that emulates or simulates the operations. A code segment may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc. The program or code segments may be stored in a processor readable medium or transmitted by a computer data signal embodied in a carrier wave, or a signal modulated by a carrier, over a transmission medium. The “processor readable or accessible medium” or “machine readable or accessible medium” may include any medium that can store, transmit, or transfer information. Examples of the processor/machine readable/accessible medium include an electronic circuit, a semiconductor memory device, a read only memory (ROM), a flash memory, an erasable ROM (EROM), a floppy diskette, a compact disk (CD-ROM), an optical disk, a hard disk, a fiber optic medium, a radio frequency (RF) link, etc. The computer data signal may include any signal that can propagate over a transmission medium such as electronic network channels, optical fibers, air, electromagnetic, RF links, etc. The code segments may be downloaded via computer networks such as the Internet, Intranet, etc. The machine accessible medium may be embodied in an article of manufacture. The machine accessible medium may include data that, when accessed by a machine, cause the machine to perform the operations described in the following. The term “data” here refers to any type of information that is encoded for machine-readable purposes. Therefore, it may include program, code, data, file, etc.
All or part of an embodiment may be implemented by software. The software may have several modules coupled to one another. A software module is coupled to another module to receive variables, parameters, arguments, pointers, etc. and/or to generate or pass results, updated variables, pointers, etc. A software module may also be a software driver or interface to interact with the operating system running on the platform. A software module may also be a hardware driver to configure, set up, initialize, send and receive data to and from a hardware device.
An embodiment may be described as a process which is usually depicted as a flowchart, a flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process is terminated when its operations are completed. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function.
The various embodiments are intended to be protected broadly within the spirit and scope of the appended claims.
Number | Date | Country | Kind |
---|---|---|---|
PCT/CN06/03614 | Dec 2006 | CN | national |