Patent Application
20230409238
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Further, it should not be assumed that any of the approaches described in this section are well-understood, routine, or conventional merely by virtue of their inclusion in this section.
As computing throughput scales faster than memory bandwidth, various techniques have been developed to keep the growing computing capacity fed with data. Processing In Memory (PIM) incorporates processing capability within memory modules so that tasks can be processed directly within the memory modules. In the context of Dynamic Random-Access Memory (DRAM), an example PIM configuration includes vector compute elements and local registers. The vector compute elements and the local registers allow a memory module to perform some computations locally, such as arithmetic computations. This allows a memory controller to trigger local computations at multiple memory modules in parallel without requiring data movement across the memory module interface, which can greatly improve performance, particularly for data-intensive workloads. Examples of data-intensive workloads include machine learning, genomics, and graph analytics.
One of the challenges with PIM is that in situations where the information required for a complete PIM command requires more bits that the command bus width, multiple command cycles are needed for each PIM command to convey the required information. For example, suppose that N number of bits is needed to specify a command, a source register, and a destination register. If the command bus width is only K number of bits and Nis greater than K, then multiple command cycles are needed for each PIM command to convey the N number of bits of command information. Requiring multiple command cycles for each PIM command increases command bus congestion, which reduces throughput and increase power consumption.
One solution for addressing this problem is to trigger complex PIM operations with a single command, where each PIM command issued from a host to memory invokes many PIM commands inside the memory module. One disadvantage to this solution is that it leads to non-deterministic timing of memory commands, which increases scheduling complexity at the memory controller. Another disadvantage is that the use of complex PIM commands prevents fine-grained interleaving of PIM and non-PIM commands, which can adversely affect the performance of co-scheduled non-PIM computations.
Another solution for addressing this problem uses processor instructions that can repeat a single instruction with incrementing operand addresses until a specified condition is satisfied, such as a count threshold, a zero/non-zero result, etc. One disadvantage to this solution is that it is only applicable to memory-to-memory string operations where a memory address is incremented, and is not applicable to incrementing source and destination registers. Yet another similar solution is vector computing architectures that increment register and operand IDs while repeating an operation, but this solution is only applicable to operations performed on entire vectors of a specified length at a host, and does not allow for fine-grained interleaving of instructions, especially in PIM. There is, therefore, a need for an approach for implementing PIM that addresses the foregoing limitations.
Implementations are depicted by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements.
In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the implementations. It will be apparent, however, to one skilled in the art that the implementations may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the implementations.
An approach is provided for processing near-memory processing commands, e.g., PIM commands, using PIM register definition data. According to an implementation, PIM register definition data defines multiple combinations of source and/or destination registers to be used to process PIM commands. A particular combination of source and/or destination registers to be used to process a PIM command is specified by the PIM command or determined by a near-memory processing element processing the PIM command. According to another implementation, the PIM register definition data specifies initial source and/or destination registers and one or more update functions for each PIM command. A near-memory processing element processes a PIM command using the initial source and/or destination registers and uses the one or more update functions to update the source and/or destination registers to be used the next time the PIM command is processed, e.g., by changing a source register value, a destination register value, or both the source register value and the destination register value. Applying an update function may, for example, increment or decrement a source or destination register value by a specified amount.
The approach harnesses commonality in source and/or destination registers among PIM commands to reduce the amount of data in PIM commands, e.g., bits, which need to be allocated to specify source and destination registers in PIM commands, and makes those bits available for other purposes. The approach eliminates the need for multiple command cycles to provide all of the information needed for a PIM command. This reduces command bus traffic and power consumption, while maintaining fine-grained control. The approach is particularly beneficial for code segments that repeat similar computations with varied operands that specify different source and/or destination registers.
Implementations are described herein in the context of PIM and PIM commands for purposes of explanation, but implementations are applicable to any type of near-memory processing technology. Implementations are also described herein in the context of near-memory registers for purposes of explanation, but implementations are applicable to any type of near-memory local storage, such as buffers, etc. As used herein, the term “near-memory” refers to anywhere within or near a memory module, such as at caches memory controllers, etc.
In step 104, the near-memory processing element determines one or more source and/or destination registers for the PIM command using PIM register definition data. In the implementation where the PIM register definition data defines multiple combinations of source and/or destination registers, the near-memory processing element uses a particular combination of source and/or destination registers as specified by the PIM command or determined by the near-memory processing element, as described in more detail hereinafter. In the implementation where the PIM register definition data specifies how to dynamically determine PIM registers, the near-memory processing element determines the initial combination of source and/or destination registers to be used to process the PIM command.
In step 106, the near-memory processing element processes the PIM command using the source and/or destination registers determined using the PIM register definition data.
In step 108, in the implementation where the PIM register definition data specifies how to dynamically determine the source and/or destination registers, the near-memory processing element updates the source and/or destination registers for the next invocation of the PIM command using the PIM register definition data, as described in more detail hereinafter.
The processor 210 is any type of processor, such as a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Logic Array (FPGA), an accelerator, a Digital Signal Processor (DSP), etc. The processor 210 includes the capability, e.g., via memory command logic, to issue near-memory processing commands, such as PIM commands. The memory module 230 is any type of memory module, such as a Dynamic Random Access Memory (DRAM) module, a Static Random Access Memory (SRAM) module, etc. According to an implementation the memory module 230 is a PIM-enabled memory module.
The memory controller 220 manages the flow of data between the processor 210 and the memory module 230 and is implemented as a stand-alone element or in the processor 210, for example on a separate die from the processor 210, on the same die but separate from the processor, or integrated into the processor circuitry as an integrated memory controller. The memory controller 220 is depicted in the figures and described herein as a separate element for explanation purposes.
The processing logic 262 processes PIM commands using the PIM register definition data 266 and is implemented by computer hardware elements, computer software, or any combination of computer hardware elements and computer software. The local storage 264 is used by the processing logic 262 for performing computations and is implemented, for example, by one or more registers, although any type of local storage may be used. The PIM register definition data 266 generally specifies combinations of source and/or destination registers to be used to process PIM commands. As described in more detail hereinafter, in one implementation the PIM register definition data 266 defines pre-defined combinations of source and/or destination registers that are selectable for use with each PIM command. According to another implementation, the PIM register definition data 266 defines an initial combination of source and/or destination registers to be used with each PIM command and one or more update functions to update the combination of source and/or destination registers to be used to process subsequent invocations of each PIM command. The PIM register definition data 266 is stored, for example, in a command buffer in the PIM execution unit 260 and is configurable. Although implementations are depicted in the figures and described herein in the context of the PIM register definition data 266 being stored within the PIM execution unit 260, implementations are not limited to this example, and the PIM register definition data may be stored external to the PIM execution unit 260, within the memory module 230 or external to the memory module 230.
A. Introduction
B. Pre-Defined Combinations of PIM Registers
According to an implementation, the PIM register definition data 266 specifies multiple pre-defined combinations of source and/or destination registers.
As another example, the first combination (Combination 1) for the pim-add command specifies that the value stored in register reg 0 is added to the value stored in register reg 1, and then the sum is stored in register reg 0. The second combination for the pim-add command specifies that value stored in register reg 2 is added to the value stored in register reg 3, and the sum is stored in register reg 2. The particular register combinations depicted in
According to an implementation, the combination of source/and or destination registers to be used is specified by the PIM command. For example, PIM commands include an operand that specifies the combination of source/and or destination registers to be used for a particular PIM command. In the example depicted in
According to another implementation, the current combination of source/and or destination registers is tracked and automatically incremented on each invocation of a PIM command. For example, the first time that a pim-load command is executed, the source and/or destination register combination specified by Combination 1 is used. The second time that the pim-load command is executed, the source and/or destination register combination specified by Combination 2 is used. This continues until the last combination of source/and or destination registers has been used and on the next invocation of the pim-load command, the current combination of source/and or destination registers “rolls over” to the first combination of source/and or destination registers and Combination 1 is used again. In this implementation, the processing logic 262 tracks the current combination of source/and or destination registers for each PIM command and advances it to the next combination upon each invocation of the corresponding PIM command. This implementation provides the additional technical benefit that PIM commands do even not need to specify the combination of source/and or destination registers to be used, which further reduces the amount of data required for a complete PIM command.
Although the example of
In step 504, the near-memory processing element determines one or more source and/or destination registers for the PIM command using PIM register definition data. For example, the processing logic 262 in the PIM execution unit 260 uses the PIM register definition data 266 to determine the particular source and/or destination registers for the combination of source/and or destination registers specified by the PIM command or determined by the processing logic 262, e.g., based upon the current combination of source and/or destination registers.
In step 506, the near-memory processing element processes the PIM command using the determined source and/or destination registers. For example, the processing logic 262 processes the PTM command using the source and/or destination registers determined in step 504.
C. Dynamically-Determined Combinations of PIM Registers
According to an implementation, combinations of source/and or destination registers are dynamically determined using the PIM register definition data 266. This includes dynamically determining the source and/or destination registers for PIM commands using update functions.
For example, the first time that a pim-load command is received for processing by the PIM execution unit 260, the processing logic 262 uses register reg0 as the destination for the value stored at location X. The processing logic 262 then uses the “Add 2” function in the “Destination Update Function” column to increment the destination register value by two for the next time that the pim-load command is processed. The processing logic 262 stores the updated destination value of register reg2 in the Destination column for the pim-load command in the table 410. Alternatively, the updated destination value is stored elsewhere, such as in the local storage 264. The source and destination register values for other PIM commands are not updated. In the example of the pim-load command, the Source 1 Update Function and the Source 2 Update Function are indicated in the table 410 as not applicable (n/a) since the source for a pim-load command is specified as an operand in the command.
The first time the pim-load command is received by the PIM execution unit 260 for processing, the processing logic 262 determines the initial value for the destination register of register reg0 from table 410. The value from location L1 in the memory 610 is loaded into register reg0. The processing logic 262 then updates the destination register value using the Destination Update Function from table 410 and adds two to the destination register value, or register reg2.
The next time the pim-load command is processed by the processing logic 262, the value from location L2 in the memory 610 is stored in reg2 and the destination register value is again incremented by two to register reg4. The third time that the pim-load command is processed, the value from location L3 in the memory 610 is stored in register reg4. These three iterations of the pim-load command are accomplished without the pim-load command having to specify the destination register for the data, or a particular combination of source and/or destination registers using an operand as previously described herein, and is very useful, for example, for workloads where a large number of values are loaded from memory into registers and then one or more computations are performed on the values stored in the registers.
Continuing with another example from the table 410 of
According to an implementation, when a change to a source or destination register value reaches the last or first register, by incrementing or decrementing, respectively, the next update causes the register value to roll over to the next available register to avoid invalid register values. For example, when the current register value is the last register, e.g., register reg9 in a 10 register implementation, and the update function specifies that the register value is to be incremented by one, the next register value rolls over to register reg0. Similarly, when the current register value is zero and the update function specifies that the register value is to be decremented by one, the next register value rolls over to register reg9.
Implementations are not limited to the example update functions depicted in
Although implementations are depicted in
The data in tables 400, 410 is presented in the figures in table format for explanation purposes only and the data in tables 400, 410 is stored in any manner and/or format that may vary depending upon a particular implementation.
In step 704, the near-memory processing element determines one or more source and/or destination registers for the PIM command using PIM register definition data. For example, the processing logic 262 in the PIM execution unit 260 identifies the current source and/or destination register values specified in table 410.
In step 706, the near-memory processing element processes the PIM command using the determined source and/or destination registers. For example, the processing logic 262 processes the PTM command using the source and/or destination registers determined in step 704.
In step 708, the near-memory processing element uses the update functions to update the source and/or destination register values for the next time that the PIM command is processed. In the present example, the processing logic 262 applies the update function(s) to determine new source and/or destination register values to be used the next time that the PIM command is processed. As previously described herein, in some implementations the update functions are applied before a PIM command is processed.
The use of pre-defined combinations of source and/or destination registers and dynamically determining combinations of source and/or destination registers described herein are not mutually exclusive and are combinable. For example, according to an implementation, the PIM register definition data 266 includes both predefined combinations of source and destination registers for some PIM commands, and dynamically determined combinations using update functions for other PIM commands. This provides great flexibility for software developers to optimize the use of these solutions for particular implementations.
According to an implementation, software support is provided for configuring and updating the PIM register definition data 266. For example, in some implementations the software support includes the capability to configure and update the data contained in the tables 400, 410. In other implementations, the software support includes, for the pre-defined combinations implementation of
