Patent Application
20130054898
The present invention relates to cache memory storage, and more particularly to a system and method for locking data in a cache memory.
A cache memory may store data (e.g., locally where it is instantly accessible to a processor) that replicates data stored in other source memories (e.g., externally where it takes many cycles for a processor to retrieve the data). If a processor requests data and the requested data is stored in the cache (e.g., a “cache hit”), the data may be loaded by simply reading the cache, which is relatively fast. Otherwise, if the requested data is not stored in the cache (e.g., a “cache miss”), the data may be fetched from its original source memory, which is relatively slow. Hence, the more requests that are served from the cache, the faster and more efficient the overall system performance.
To transfer new data into the cache memory, data previously stored in the cache may be erased to make room for the new data. In some cases, a set of data may be used repeatedly by a program or application. In such cases, it may be inefficient to repeatedly load and erase the same data in the cache over and over again for each repeated use. Some current systems implement a lock mechanism to “lock” a set of frequently used data into the cache so that it may not be replaced and may be re-used to avoid redundant data transfer. Typically, an entire line of data is locked together in the cache.
Some systems may lock data after the data has already been loaded in the cache. Such systems typically stall a program, preventing the program from accessing the requested data already loaded into the cache, until the lock is executed. The lock mechanism may search all entries in the cache until the target data is found and locked, which may cause the program to incur an interruption penalty of several computational cycles.
To avoid stalling the program, other systems may lock data in a cache only prior to running the program. The cache memory may be emptied (invalidated) and data may be loaded and locked exclusively into the first available slot(s) of the cache prior to running the program. Although such systems avoid lock interruptions that stall a running program, such systems cannot benefit from the lock mechanism during program operation, thereby using extra computational cycles to re-transfer frequently used data into the cache for each repeated use.
The subject matter regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of operation, together with objects, features, and advantages thereof, may best be understood by reference to the following detailed description when read with the accompanying drawings. Specific embodiments of the present invention will be described with reference to the following drawings, wherein:
It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements.
In the following description, various aspects of the present invention will be described. For purposes of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the present invention. However, it will also be apparent to one skilled in the art that the present invention may be practiced without the specific details presented herein. Furthermore, well known features may be omitted or simplified in order not to obscure the present invention.
Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification discussions utilizing terms such as “processing,” “computing,” “calculating,” “determining,” or the like, refer to the action and/or processes of a computer or computing system, or similar electronic computing device, that manipulates and/or transforms data represented as physical, such as electronic, quantities within the computing system's registers and/or memories into other data similarly represented as physical quantities within the computing system's memories, registers or other such information storage, transmission or display devices.
Conventional cache locking systems are typically software implemented and therefore compete with a software program for processor resources used to load the cache. Some systems set the locking address and lock data prior to running the program, and therefore use processor resources before the program has started, while other systems set the locking address and lock data while the program is running and therefore use processor resources while the program is running, causing the program to stall. Regardless of when the lock is executed, these conventional cache locking systems compete with the program for processor resources and may therefore limit program performance.
According to some embodiments of the invention, a system and method is provided for a hardware implemented locking system, which does not compete with a software program for processor resources and may therefore operate to lock data at any time while the program is running without stalling or stopping the program (e.g., with zero computational penalty). Furthermore, instead of loading and locking data in separate operations, embodiments of the invention may lock data as it is being loaded into the cache memory, “on-the-fly,” as part of the data transfer operation. As each new set of requested data is being loaded from a source memory into the cache, a hardware module, e.g., in a cache controller, may lock data in the cache simultaneously or in the same computational cycle.
In another embodiment, instead of operating the locking mechanism separately using a hardware module, some systems may use two separate software threads operated independently of each other, one to run the program (e.g., to retrieve requested data from the cache) and another to operate the lock (e.g., to lock the requested data in the cache). The separate program thread and lock thread may operate independently, for example, using different cores of a multi-core processor, to lock data on-the-fly.
The program thread request for data may trigger a load module, e.g., a cache controller, to load the requested data. In one embodiment, the lock module may be integral to or coupled with the load module (e.g., when both modules are either hardware or software implemented). In such embodiments, these integrated modules may be configured to load and lock data simultaneously. However, in another embodiment, these modules may be separate (e.g., when one module is hardware implemented and the other module is software implemented). In such embodiment, an additional synchronization mechanism may be used to coordinate these modules to simultaneously lock and load the data.
To load the requested data, the load module may read load attributes, for example, from a memory management table, such as, an address control unit (ACU) or memory management unit (MMU). The load attributes may include information for loading the requested data, such as, the address(es) of the requested data in a source memory, whether or not the requested data is cacheable or not cacheable, memory access attributes, etc. The lock module (either hardware or software implemented and operating independently of the program) may use a new attribute: a “lock” attribute to determine whether or not to lock the requested data. When the lock attribute indicates a “yes” (e.g., an attribute bit value of (1)), the data may be pre-designated to be locked and when the lock attribute indicates a “no” (e.g., an attribute bit value of (0)), the data may be pre-designated not to be locked. The lock attribute may be stored together with or separate from the load attributes, for example, in a look-up-table, such as an ACU, MMU, a data register, a data buffer, or may in the program load instructions themselves.
Once the lock module determines whether or not to lock data (e.g., according to the pre-defined lock attribute for the data), the lock module may lock the data as the load mechanism loads the data into the cache, for example, in the same computational cycle. The lock module may lock data loaded in an entry of the cache by setting a lock value to “lock” in a cache table corresponding to the entry. The lock values in the cache table may match the lock attribute values in the data management table for the loaded entries.
Reference is made to
Device 100 may include a computer device, cellular device, or any other digital device such as a cellular telephone, personal digital assistant (PDA), video game console, etc. Device 100 may include any device capable of executing a series of instructions to run a computer program. Device 100 may include an input device 101 for receiving input from a user (e.g., via a pointing device, click-wheel or mouse, keys, touch screen, recorder/microphone using voice recognition, etc.) and an output device 102 (e.g., a monitor or screen) for displaying a program on a user interface according to a sequence of instructions executed by a processor 1.
Processor 1 may include a central processing unit (CPU), a digital signal processor (DSP), a microprocessor, a controller, a chip, a microchip, a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC) or any other integrated circuit (IC), or any other suitable multi-purpose or specific processor or controller.
Processor 1 may be coupled to data memory unit 2 via a data bus 4 and to program memory unit 3 via a program memory bus 5.
Program memory unit 3 may store a set of instructions and rules for running a computer program while data memory unit 2 may store data used for running the program, such as, pre-generated (i.e. static) data and/or scratch pad (i.e. dynamic) data generated while operating the program instructions; however, other arrangements for storing instructions and data in a memory or memories may be used.
Data memory unit 2 and program memory unit 3 may include, for example, static random access memory (SRAM), dynamic RAM (DRAM), flash memory, or other suitable memory units or storage units, and may use volatile memory or non-volatile memory. Data memory unit 2 and program memory unit 3 may be implemented as separate or integrated memory units. Data memory unit 2 and/or program memory unit 3 may be integrated (“on-chip”) or may be separate (i.e., “off-chip”). In one embodiment in which there is a multi-level memory or a memory hierarchy, program memory unit 3 may be off-chip and data memory unit 2 may be on-chip, e.g., an L-1 cache (e.g., which is relatively more integrated with processor 1 than an L-2 cache and typically runs at the clock rate of processor 1) or an L-2 cache (e.g., which is relatively less integrated than the L-1 cache and typically runs at a different rate than the clock rate of processor 1). Other or additional memory architectures may be used.
Cache memory 10 may be a volatile memory, for example, including one or more blocks or lines of data requested, or likely to be requested, by a program or processor 1. Cache memory 10 may replicate data stored in data memory unit 2 (or program memory 3), where processor 1 may retrieve the data relatively faster and at a relatively lower computational cost from cache memory 10 than from data memory unit 2 (or program memory 3). Cache memory 10 may include central processing unit (CPU) cache, disk cache, web cache, or any other type of buffer or cache. Although the example of
Processor 1 may include a program control unit 8. Program control unit 8 may request, retrieve, and dispatch instructions from program memory unit 3 and may be responsible, in general, for the program pipeline flow. Program control unit 8 may be coupled to a data memory controller 16 via data memory bus 4, and a program memory controller 18 coupled to program memory bus 5 to retrieve data from data memory unit 2 and program memory unit 3, respectively.
Processor 1 may include one or more fetch unit(s) 12 and dispatch unit(s) 6. Fetch unit 12 may fetch a set (e.g., a line) of instructions from program memory 3 in each cycle. Once the instructions are fetched, program control unit 8 may read and extract instruction packets from the fetched instructions and send the instructions to dispatch unit 6. Dispatch unit 6 may dispatch and decode the instructions. Execution unit(s) 11 may execute instructions. Some instructions may request data from data memory unit 2. Load store unit (LSU) 7 may retrieve the requested data, for example, locally from cache memory 10 (relatively quickly) or externally from the original source data memory unit 2 (relatively slowly).
The data requested by the instructions may be retrieved prior to executing the instructions so that the data is available therefore. Data memory controller 16 may load and store the requested data from data memory unit 2 into cache memory 10, where the data may be quickly retrieved for use by processor 1. Data memory controller 16 may be coupled to one or more data management table(s) 15 (e.g., a look-up-table, an ACU, a MMU). Data management table(s) 15 may store load attributes defining information for loading each set of requested data, for example, address(es) of the requested data in data memory unit 2, whether or not the data is cacheable, etc. When cache memory 10 is full, data memory controller 16 may invalidate or empty cache memory 10 to make space to load new data.
A lock module 14 may lock data into cache memory 10 that is frequently used by the program to prevent the data from being replaced when cache memory 10 is emptied. The locked data may remain in cache memory 10 for repeated use by the program. The program may run in a processing thread (e.g., implemented by a software module) and the lock mechanisms may run independently in a separate processing thread (e.g., implemented by a hardware module or a second separate software module). Accordingly, running the lock mechanism may not interrupt or stall the program.
Data memory controller 16 may determine if a set of requested data is already stored and/or locked in cache memory 10. If the requested data is already stored or locked in cache memory 10, the program may retrieve the data directly from cache memory 10, instead of the relatively slower data memory unit 2. However, if the requested data is not stored or locked in cache memory 10, the program may load the data from the original source location in data memory unit 2.
In a separate lock processing thread, for example, on-the-fly or while the data is being loaded from data memory unit 2 into cache memory 10, lock mechanism 14 may determine whether or not to lock the new retrieved data in cache memory 10 and, if so, may execute the lock. Lock mechanism 14 may determine whether or not to lock the data by reading a pre-defined value for a lock attribute associated with the data. A first value (e.g., numeral, numeral range, marker, flag, register state or index) for the lock attribute may indicate that the associated retrieved data should be locked and a second value may indicate that the associated retrieved data should not be locked. The lock attribute values for each set of requested data may be stored in data management table 15, such as, a look-up-table, a memory management buffer, data registers, or may be defined in the load instruction(s) requesting the data. The lock attribute values for each set of requested data may be pre-designated to initiate locking or no locking, for example, based on the frequency or rate of requests for the data in the program, program segment, or line of instructions. If the lock attribute value in data management table 15 indicates the data is to be locked, lock mechanism 14 may set a corresponding lock value in a cache table 13 to a lock setting, thereby locking the data in the associated entry in cache memory 10. For a no lock attribute value, lock mechanism 14 may set a corresponding lock value in a cache table 13 to a no lock setting or may not change the lock value from a default no lock value. Lock mechanism 14 may lock the data in a separate processing thread independently of the program thread using the locked data, for example, to seamlessly lock the data without stalling the program operation.
In some embodiment, instead of dividing loaded data into two categories, one to be locked and another not to be locked, some systems may determine a measure, score, rank, preference or degree of locking along a continuous or finite scale. The measure of locking may be proportional to the frequency or rate of requests for the data in the program. For example, as the frequency or rate of requests for data increases, the locking measure for the data may also increase. Locking mechanism 14 may lock data having an above threshold locking measure. Alternatively, locking mechanism 14 may lock a predetermined amount of data (e.g., into a data bock in cache memory 10 or a separate cache of fixed size pre-designated to store locked data), for example, including data having the highest locking measure relative to other requested data. The predetermined locking threshold or amount of locked data may be set or adjusted, for example, automatically by a processor or by a user, to adjust the amount of locked data or frequency requirement for data to be locked in cache memory 10.
Embodiments of the invention provide a system and method for seamlessly locking data in cache memory 10 while running a program using the same data and without incurring a computational penalty to the program for locking the data.
Lock module 14 may be configured in various hardware and/or software implementations in device 100. In one embodiment of a hardware implementation shown in
Reference is made to
In operation 200, a processor (e.g., processor 1 of
In operation 210, the data memory controller may determine if the requested data is already stored in the cache. If the requested data is already stored in the cache, a process or processor may proceed to operation 220 to request and retrieve the data from the cache. However, if the requested data is not stored in the cache, a process or processor may proceed to operation 230 to attempt to replace the non-requested data stored in the cache with the requested data.
In operation 230, the data memory controller may determine if the data stored in the cache is locked. If the data is locked in the cache, the data cannot be replaced with the requested data and a process or processor may attempt to replace another (sequential) cache entry or proceed to operation 240 to abandon cache storage, for example, transferring the requested data directly from an original data source memory location (e.g., data memory unit 2 of
In operation 250, the data memory controller may retrieve the requested data from the source memory location, for example, via an external program port (EPP).
In operation 260, the data memory controller may fill the (un-locked) cache location with the requested data retrieved from the source memory. The requested data may simultaneously be served to the processor to complete the data request.
In operation 270, a lock module (e.g., hardware or software implemented lock module 14 of
In operation 280, the lock module may lock the requested data into the cache. The cache may be associated with a cache table (e.g., cache table 13 of
Other operations or orders of operations may be used.
In some embodiments, the lock module and load module of
Reference is made to
In operation 300, a processor (e.g., processor 1 of
In operation 310, the processor may initiate a lock module in a second processing thread using a hardware module or a second software module to determine whether or not to lock the requested data in the cache. The second processing thread (for the lock) may be operated independently of the first processing thread (for the program) so that the lock module does not stall the program. The second processing thread may be initiated if the requested data is not already stored or locked in the cache; otherwise the requested data may simply be retrieved from the cache.
In operation 320, the processor may read one or more lock attribute value(s) associated with the requested data that indicate whether or not to lock the requested data in the cache. The lock attribute value may be stored in a look-up-table (e.g., data management table 15 of
In operation 330, if the requested data is determined to be locked, the processor may lock the requested data in the cache at the same time as the data is loaded into the cache. Otherwise the processor may store, but not lock, the data.
Other operations or orders of operations may be used.
As it is used herein, a processing “thread” may refer to any sequence of operations or steps, which may be implemented by either hardware or software modules.
As it is used herein, locking and storing/transferring data “at the same time” may refer to locking and storing/transferring data simultaneously to, parallel to, in the same clock cycle as, in sequential/consecutive/overlapping clock cycles or independently.
As it is used herein, locking data “on-the-fly” may refer to locking data seamlessly with, independently of, parallel to, without interruption to, with zero or a below threshold computational penalty to, or so as not to stall or stop, a program requesting the data.
It should be appreciated by a person skilled in the art that although data is described to be locked in a cache memory, embodiments of the invention include locking data in any other type of memory including volatile memory, non-volatile memory, dynamic or static memory, buffers, registers, tables, etc.
It should be appreciated by a person skilled in the art that data may be requested, loaded, stored and/or locked in any size group of individual or multiple data elements in a memory, such as, a line, row, column, data block, single data element, consecutive data elements, non-consecutive data elements, etc.
It should be appreciated by a person skilled in the art that the data referred to in embodiments of the invention may represent information related to any physical or virtual structure, such as, for example, video, image, audio or text data, statistical data, data used for running a program including static and/or dynamic data, etc.
Embodiments of the invention may include an article such as a computer or processor readable non-transitory medium, or a computer or processor storage medium, such as for example a memory, a disk drive, or a USB flash memory, encoding, including or storing instructions which when executed by a processor or controller (for example, processor 1 of
Although the particular embodiments shown and described above will prove to be useful for the many distribution systems to which the present invention pertains, further modifications of the present invention will occur to persons skilled in the art. All such modifications are deemed to be within the scope and spirit of the present invention as defined by the appended claims.
