Patent Application
20170024145
A system-on-a-chip (SoC) commonly includes one or more processing devices, such as central processing units (CPUs) and cores, as well as one or more memories and one or more interconnects, such as buses. A processing device may issue a data access request to either read data from a system memory or write data to the system memory. For example, in response to a read access request, data is retrieved from the system memory and provided to the requesting device via one or more interconnects. The time delay between issuance of the request and arrival of requested data at the requesting device is commonly referred to as “latency.” Cores and other processing devices compete to access data in system memory and experience varying amounts of latency.
Caching is a technique that may be employed to reduce latency. Data that is predicted to be subject to frequent or high-priority accesses may be stored in a cache memory from which the data may be provided with lower latency than it could be provided from the system memory. As commonly employed caching methods are predictive in nature, an access request may result in a cache hit if the requested data can be retrieved from the cache memory or a cache miss if the requested data cannot be retrieved from the cache memory. If a cache miss occurs, then the data must be retrieved from the system memory instead of the cache memory, at a cost of increased latency. The more requests that can be served from the cache memory instead of the system memory, the faster the system performs overall.
Although caching is commonly employed to reduce latency, caching has the potential to increase latency in instances in which requested data too frequently cannot be retrieved from the cache memory. Display systems are known to be prone to failures due to latency. “Underflow” is a failure more that refers to data arriving at the display system too slowly to fill the display in the intended manner.
One known solution that attempts to mitigate the above-described problem in display systems is to increase the sizes of buffer memories in display and camera system cores. This solution comes at the cost of increased chip area. Another known solution that attempts to mitigate the problem is to employ faster memory. This solution comes at costs that include greater chip area and power consumption.
Systems, methods, and computer programs are disclosed for reducing latency in a system that includes a system memory and a cache memory.
In an exemplary method, a pre-fetch command that identifies requested data is received from a requestor device. The requested data is pre-fetched from the system memory into the cache memory in response to the pre-fetch command. A data access request corresponding to the pre-fetch command is then received, and in response to the data access request the data is provided from the cache memory to the requestor device. The data pre-fetch may be preceded by a pre-fetch of an address translation.
An exemplary system includes a processor system, a system memory, and a cache memory. The processor system is configured with logic to receive from a requestor device a pre-fetch command that identifies requested data. The processor system is further configured with logic to pre-fetch the requested data from the system memory into the cache memory in response to the pre-fetch command. The processor is further configured with logic to respond to a data access request corresponding to the pre-fetch command by providing the data from the cache memory to the requestor device. The data pre-fetch may be preceded by a pre-fetch of an address translation.
An exemplary computer program product includes computer-executable logic embodied in a non-transitory storage medium. Execution of the logic by the processor configures the processor to: receive a pre-fetch command identifying requested data from the requestor device; pre-fetch the requested data from the system memory into the cache memory in response to the pre-fetch command; and respond to a data access request corresponding to the pre-fetch command by providing the requested data from the cache memory to the requestor device. The data pre-fetch may be preceded by a pre-fetch of an address translation.
In the Figures, like reference numerals refer to like parts throughout the various views unless otherwise indicated. For reference numerals with letter character designations such as “102A” or “102B”, the letter character designations may differentiate two like parts or elements present in the same Figure. Letter character designations for reference numerals may be omitted when it is intended that a reference numeral to encompass all parts having the same reference numeral in all Figures.
The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.
The terms “component,” “database,” “module,” “system,” and the like are intended to refer to a computer-related entity, either hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a computing device and the computing device may be a component. One or more components may reside within a process and/or thread of execution, and a component may be localized on one computer and/or distributed between two or more computers. In addition, these components may execute from various computer readable media having various data structures stored thereon. The components may communicate by way of local and/or remote processes, such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the Internet with other systems by way of the signal).
The term “application” or “image” may also include files having executable content, such as: object code, scripts, byte code, markup language files, and patches. In addition, an “application” referred to herein, may also include files that are not executable in nature, such as documents that may need to be opened or other data files that need to be accessed.
The term “content” may also include files having executable content, such as: object code, scripts, byte code, markup language files, and patches. In addition, “content” referred to herein, may also include files that are not executable in nature, such as documents that may need to be opened or other data files that need to be accessed.
The term “task” may include a process, a thread, or any other unit of execution in a device.
The term “virtual memory” refers to the abstraction of the actual physical memory from the application or image that is referencing the memory. A translation or mapping may be used to convert a virtual memory address to a physical memory address. The mapping may be as simple as 1-to-1 (e.g., physical address equals virtual address), moderately complex (e.g., a physical address equals a constant offset from the virtual address), or the mapping may be complex (e.g., every 4 KB page mapped uniquely). The mapping may be static (e.g., performed once at startup), or the mapping may be dynamic (e.g., continuously evolving as memory is allocated and freed).
In this description, the terms “communication device,” “wireless device,” “wireless telephone”, “wireless communication device,” and “wireless handset” are used interchangeably. With the advent of third generation (“3G”) wireless technology and four generation (“4G”), greater bandwidth availability has enabled more portable computing devices with a greater variety of wireless capabilities. Therefore, a portable computing device may include a cellular telephone, a pager, a PDA, a smartphone, a navigation device, or a hand-held computer with a wireless connection or link.
As illustrated in
The terms “upstream” and “downstream” may be used for convenience to reference information flow among the elements of processing system 100. The terms “master” and “slave” may be used for convenience to refer to elements that respectively initiate requests and respond to requests. Elements of processing system 100 are characterized by either a master (“M”) manner of coupling to a downstream device, a slave (“S”) manner of coupling to an upstream device, or both. It should be understood that the arrows shown in
CPU 102 includes a memory management unit (“MMU”) 112. MMU 112 comprises logic (e.g., hardware, software, or a combination thereof) that performs address translation for CPU 102. Although for purposes of clarity MMU 112 is depicted in
Processing system 100 also includes a system MMU (“SMMU”) 114. An SMMU provides address translation services for upstream device traffic in much the same way that a processor's MMU, such as MMU 112, translates addresses for processor memory accesses. SMMU 114 includes or is coupled to one or more translation caches 116. Although not illustrated in
The main functions of MMU 112 and SMMU 114 include address translation, memory protection, and attribute control. Address translation is a method by which an input address in a virtual address space is translated to an output address in a physical address space. Translation information is stored in translation tables that MMU 112 or SMMU 114 references to perform address translation, such as a translation table 118 stored in system memory 106. There are two main benefits of address translation. First, address translation allows a processing device to address a large physical address space. For example, a 32 bit processing device (i.e., a device capable of referencing 232 address locations) can have its addresses translated such that the processing device may reference a larger address space, such as a 36 bit address space or a 40 bit address space. Second, address translation allows processing devices to have a contiguous view of buffers allocated in memory, despite the fact that memory buffers are typically fragmented, physically non-contiguous, and scattered across the physical memory space.
Translation table 118 contains information necessary to perform address translation for a range of input addresses. Although not shown in
The process of traversing translation table 118 to perform address translation is known as a “translation table walk.” A translation table walk is accomplished by using a sub-segment of an input address to index into the translation sub-table, and finding the next address until a block descriptor is encountered. A translation table walk comprises one or more “steps.” Each “step” of a translation table walk involves: (1) an access to translation table 118, which includes reading (and potentially updating) it; and (2) updating the translation state, which includes (but is not limited to) computing the next address to be referenced. Each step depends on the results from the previous step of the walk. For the first step, the address of the first translation table entry that is accessed is a function of the translation table base address and a portion of the input address to be translated. For each subsequent step, the address of the translation table entry accessed is a function of the translation table entry from the previous step and a portion of the input address.
The following exemplary method for reading data 120 from system memory 106 with minimal latency is described with reference to the flow diagram of
The pre-fetch command identifies data requested by the requestor device. To identify data, the pre-fetch command may indicate an address of requested data. Alternatively, the pre-fetch command may indicate a pattern of addresses. The multiple addresses indicated by such a pattern may or may not be contiguous. The pattern thus corresponds to an amount of requested data. In an embodiment (not shown) in which there is no SMMU upstream of a requestor device or MMU associated with a requestor device, the address or address pattern indicated by the pre-fetch command may be a physical address of requested data 120 in system memory 106. However, in the exemplary embodiment shown in
In response to receiving the pre-fetch command, MMU 112 or SMMU 114 may first determine whether the one or more address translations implicated by the address indicated in the pre-fetch command are already accessible (e.g., stored in translation cache 116). If the one or more address translations are not already accessible, then MMU 112 or SMMU 114 accesses translation table 118 or system cache 108 and performs address translation methods in the manner described above, as may be needed to make the address translations accessible. For example, SMMU 114 may store the resulting address translation in translation cache 116.
As indicated by block 206, requested data 120 is then pre-fetched from system memory 106 into system cache 108. Although in the exemplary embodiment shown in
As indicated by block 208, a data access request is received from the requestor device. The data access request corresponds to the pre-fetch command. That is, for each data access request that the requestor device issues, the requestor device also issues a corresponding pre-fetch command. Although in the exemplary embodiments there is a one-to-one correspondence between data access requests and pre-fetch commands, in other embodiments there can be other relationships between data access requests and pre-fetch commands. In response to the data access request, the requested data 120 is provided from cache memory 108 to the requestor device.
The above-described exemplary method exploits the fact that in some types of processing systems, the address pattern in which the relevant data is stored is available to the requestor device well in advance of the time at which the data needs to be processed. For example, core 104 may be included in a display processing system that displays data on a display screen (not shown in
It follows from the above that it may be advantageous for a requestor device to issue the pre-fetch command a sufficient amount of time in advance of the corresponding data access request to allow the requested data 120 to become available in system cache 108 for immediate transfer to the requestor device in response to the data access request. However, it may be disadvantageous for a requestor device to issue the pre-fetch command so far in advance of the corresponding data access request that the likelihood of the data being overwritten or evicted from system cache 108 is increased.
The above-described method not only reduces latency but also may be used to promote power conservation. A requestor device, such as CPU 102 or core 104, may instruct DRAM controller 109 and other circuitry associated with system memory 106 to enter a low-power mode after pre-fetching a block of requested data 120 from system memory 106 into system cache 108.
Further details of the above-referenced address translation and information flows may be appreciated in the following exemplary method, which is described with reference to the flow diagram of
As indicated by block 306, SMMU 114 may receive the pre-fetch command or data access request generated by core 104. Block 306 exemplifies a time delay between elements. As described below, the method may promote reduction in certain time delays and thus overall latency. This particular time delay between the time at which core 104 generates a pre-fetch command or data access request and the time at which SMMU 114 receives the pre-fetch command or data access request may be referred to herein as “a0” and is considered in further detail below.
It should be noted that SMMU 114 responds to a pre-fetch command in a manner similar to that in which it responds to a data access request. However, SMMU 114 does not return the requested data to core 104 in response to a pre-fetch command. Rather, the pre-fetch command results in the requested data being made available in system cache 108. It is not until SMMU 114 receives the data access request corresponding to an earlier pre-fetch command that SMMU 114 responds by providing the requested data from system cache 108 to core 104.
As indicated by block 308, SMMU 114 determines whether the address translation needed to access the requested data is available in translation cache 116. A determination that the address translation is not available in translation cache 116 may be referred to as an MMU translation cache miss. If it is determined that such an MMU translation cache miss occurred, then it is determined whether the address translation is available in system cache 108, as indicated by block 310. The time delay for the determination that a translation cache miss occurred to trigger a search of system cache 108 for the address translation may be referred to herein as “b0.” A determination that an address translation is not available in system cache 108 may be referred to as a system cache miss. If it is determined (block 310) that a system cache miss did not occur, then the address translation is returned to SMMU 114 (i.e., to translation cache 116), as indicated by block 312. The time delay for the address translation to be returned to SMMU 114 may be referred to herein as “b1.”
If it is determined (block 310) that a system cache miss occurred, then an address translation method is begun by accessing translation table 118 in system memory 106, as indicated by block 314. The time delay for the determination that a system cache miss occurred to trigger SMMU 114 to access translation table 118 may be referred to herein as “c0.” The translation table entry obtained from translation table 118 is then stored in translation cache 116 for use by SMMU 114 in the address translation method. The time delay for the translation table entry to be stored in translation cache 116 is “b1” plus an additional delay “c1.” Note that SMMU 114 may generate multiple accesses of translation table 118 in association with performing the address translation method.
If it is determined (block 308) that no translation cache miss occurred, then it is determined whether the requested data 120 is available in system cache 108 as indicated by block 316. As stated above, the time delay for a determination that a translation cache miss occurred to trigger a search of system cache 108 is “b0.” If it is determined (block 316) that no system cache miss occurred, then the requested data 120 is returned to core 104, as indicated by block 318. However, if it is determined (block 316) that a system cache miss occurred, then the requested data 120 must be read from system memory 106 into system cache 108, as indicated by block 320. Although in the exemplary embodiment such requested data 120 may be read into system cache 108, it should be understood that in other embodiments the requested data alternatively may be transferred directly to the core or other requestor device without storing it in system cache. For example, display data may be transferred directly to a core that requested the display data, since display data is generally not reused.
The time delay for the determination that a system cache miss occurred to trigger SMMU 114 to access system memory 106 is “c0.” The time delay for the requested data 120 to be read from system memory 106 into system cache 108 is “c1.” The requested data 120 is then returned to core 104, as indicated by block 318. The time delay for requested data 120 to traverse SMMU 114 and reach core 104 may be referred to as “a1.”
In the absence of a pre-fetch command, the total time delay or access time (“T”) between core 104 issuing a data access request and the requested data 120 being returned to core 104 is: T=a0+Mmiss*(b0+b1+c0+c1)+Mhit*(b0+b1)+b0+c0+b1+c1+a1, where “Mmiss” is the number of accesses generated by SMMU 114 to obtain the translation table entry that resulted in a system cache miss, “Mhit” is the number of accesses generated by SMMU 114 to obtain the translation table entry that resulted in a system cache hit, and where Mmiss>=0, and Mhit>=0.
However, by core 104 issuing a pre-fetch command an optimal amount of time in advance of issuing a data access request, then the requested data 120 will be available in system cache 108 for immediate access by core 104, thus reducing the total delay to: T′=a0+b0+b1+a1. This assumes that the translation table entry is also prefetched in the MMU ahead of time.
Processing system 100 (
A video encoder 414, e.g., a phase-alternating line (“PAL”) encoder, a sequential couleur avec memoire (“SECAM”) encoder, a national television system(s) committee (“NTSC”) encoder or any other video encoder, is coupled to CPU 404. Further, a video amplifier 416 is coupled to the video encoder 414 and the touchscreen display 412. A video port 418 is coupled to the video amplifier 416. A USB controller 420 is coupled to CPU 404. A USB port 422 is coupled to the USB controller 420. A memory 424, which may operate in the manner described above with regard to system memory 106 (
A stereo audio CODEC 430 may be coupled to the analog signal processor 406. Also, an audio amplifier 432 may be coupled to the stereo audio CODEC 430. In an exemplary aspect, a first stereo speaker 434 and a second stereo speaker 436 are coupled to the audio amplifier 432. In addition, a microphone amplifier 438 may be coupled to the stereo audio CODEC 430. A microphone 440 may be coupled to the microphone amplifier 438. A frequency modulation (“FM”) radio tuner 442 may be coupled to the stereo audio CODEC 430. Also, an FM antenna 444 is coupled to the FM radio tuner 442. Further, stereo headphones 446 may be coupled to the stereo audio CODEC 430.
A radio frequency (“RF”) transceiver 448 may be coupled to the analog signal processor 406. An RF switch 450 may be coupled between the RF transceiver 448 and an RF antenna 452. The RF transceiver 448 may be configured to communicate with conventional terrestrial communications networks, such as mobile telephone networks, as well as with global positioning system (“GPS”) satellites.
A mono headset with a microphone 456 may be coupled to the analog signal processor 406. Further, a vibrator device 458 may be coupled to the analog signal processor 406. A power supply 460 may be coupled to the on-chip system 402. In a particular aspect, the power supply 460 is a direct current (“DC”) power supply that provides power to the various components of the portable communication device 400 that require power. Further, in a particular aspect, the power supply is a rechargeable DC battery or a DC power supply that is derived from an alternating current (“AC”) to DC transformer that is connected to an AC power source.
A keypad 454 may be coupled to the analog signal processor 406. The touchscreen display 412, the video port 418, the USB port 422, the camera 428, the first stereo speaker 434, the second stereo speaker 436, the microphone 440, the FM antenna 444, the stereo headphones 446, the RF switch 450, the RF antenna 452, the keypad 454, the mono headset 456, the vibrator 458, and the power supply 460 are external to the on-chip system 402.
One or more of the method steps described herein (such as described above with regard to
Alternative embodiments will become apparent to one of ordinary skill in the art to which the invention pertains without departing from its spirit and scope. Therefore, although selected aspects have been illustrated and described in detail, it will be understood that various substitutions and alterations may be made therein without departing from the spirit and scope of the present invention, as defined by the following claims.
