This invention relates to the partitioning of computer memory, in particular within a processor cache.
Caching is a well-known technique for improving computer performance and different types of caches are found in almost every modern computer. A processor, that is, CPU cache is usually a relatively small but fast hardware memory structure in which copies of frequently needed information (instructions and data) are stored so as to be more readily accessible. Traditionally, computer processors have employed a simple mapping from physical memory addresses to processor cache sets, in which the low-order bits of the physical page number are used as the high-order bits of the cache set index. The term “page” generally refers to a contiguous, aligned region of memory, and is typically used as a unit for address translation and memory management. For example, systems having the x86 architecture commonly use 4 KB pages.
A hardware “cache set” contains space for caching a limited number of memory units, typically referred to as cache “lines”. For example, on modern x86 hardware, the cache line granularity is 64 bytes, with 64-byte alignment. On the Intel Sandy Bridge x86 processor, a single last-level cache set consists of 20 lines; i.e., the cache is 20-way set associative.
The traditional, straightforward hardware mapping of physical pages to cache sets has been leveraged for many years by operating systems and hypervisors, using a well-known technique known as “page coloring”. Pages are partitioned into disjoint sets called “colors”, such that pages with different colors do not conflict in the cache. A page's color can be computed trivially from its physical address, for example, using a simple shift-and-mask technique. Page coloring has been used in many systems to improve performance by reducing cache conflict misses and to control the isolation or sharing of cache memory between software contexts.
However, some recent processors, such as those based on the Intel Sandy Bridge (SNB) and Ivy Bridge (IVB) x86 micro-architectures, now use “complex cache indexing” to map physical addresses to cache sets in the processor's last level cache (LLC). The hardware that realizes this mapping can be implemented using an arbitrarily complicated, undocumented, proprietary hash function that may potentially use any of the bits in the physical memory address to index into the cache. As a result, small contiguous memory regions may be scattered across many discontiguous sets throughout the cache, and traditional page coloring techniques may no longer work. The mapping function may also vary across different processor implementations or configurations, even within the same processor family.
It would therefore be advantageous to have an automated software-based approach that can partition memory units (such as pages or lines) into sets, such that units in different sets do not contend for the same limited space within the processor cache. Preferably, this method should work even for processors that employ opaque complex cache indexing to map physical addresses to their corresponding cache sets. Such a capability would enable software, including operating systems and hypervisors, to manage or eliminate cache conflict misses by consulting this partitioning when making memory management decisions, enjoying benefits similar to traditional page coloring. This capability is especially useful in the context of a software cryptoprocessor system, such as the vCage system provided by PrivateCore, Inc., in which the ability to control cache residency and prevent evictions helps maintain confidentiality and integrity.
System-level software such as an operating system (OS) and/or hypervisor 2000 will typically be included to perform well-known functions. Various system- and user-level processes 6200 run under the supervision of the OS/hypervisor 2000. In some systems, virtual machines (VMs) 6300, which can be considered a special type of processes, also run on the system software. A hypervisor will typically not be necessary if no virtual machines are included; both options are illustrated in
In
Of particular interest in the context of this invention is the cache 5000, which is part of the CPU 1000. Some systems use a multi-level cache. In a two-level (L1, L2) cache, the L1 cache is typically faster but smaller and is ordinarily included within the processor itself. The L2 cache will generally be larger but slower than the L1 cache and may come between the L1 cache and system memory. Multi-level cache systems generally operate by checking the L1 cache first; if it hits, the processor proceeds at high speed. If the smaller L1 cache misses, L2 is checked, and so on, before external memory is checked. Some systems have additional cache levels, and still others have separate L1 caches for instructions and data. The general structure and properties of a cache are well-understood in the field of computer science and will therefore not be described further here, except to the extent needed or helpful to understand the different embodiments.
A cache partitioning software module (the “cache partitioner”) 3000 is included either at the application level, as shown in
According to various embodiments, the cache partitioner partitions memory units into non-conflicting sets at cache-line granularity, and executes on the processor 1000 to discover its address-to-set mapping dynamically. No up-front knowledge of the processor cache organization is required; however, if partial information or constraints regarding the mapping happen to be known, they can be leveraged to accelerate the process. In one embodiment, each line within a specified physical memory region may be partitioned into disjoint sets, so that addresses in different sets do not conflict in the Last Level Cache (LLC), for example, the L1 cache 5000.
A collection of physical addresses (PAs) forms an input to the cache partitioner at cache-line granularity, for example, 64-byte alignment on x86 hardware. In one embodiment, the input addresses may be specified conveniently as a single contiguous address range, but this is not required; the input may consist of multiple ranges, or even an arbitrary collection of individual line addresses. The selection of such “test” input addresses may be made by a user, for example, to test some known range so as to increase performance, or automatically; for example, the OS could designate the addresses containing all or some portion of performance-critical code for cache partitioning and optimization before the code needs to be executed, possibly repeatedly. As output, a mapping module 3100 within the cache partitioner maps each cache-line-sized input address to its corresponding cache set. This generates a partitioning of the input addresses, such that addresses identified as belonging to the same partition conflict in the cache, and addresses in different partitions do not conflict in the cache.
The method exploits the limited associativity of a single cache set to force observable cache evictions and discover conflicts.
By computing a single conflict set, the hardware cache set associativity is forcibly exceeded, which allows the cache partitioner to identify conflicting addresses by monitoring cache evictions. The degree of cache associativity may be known a priori; for example, the Intel Sandy Bridge x86 processor LLC is known to have 20-way set associativity; this information is well-documented. Associativity may in other cases be determined by querying the processor directly, for example, via the x86 CPUID instruction.
The subset of input memory addresses which may potentially conflict is first identified (
The conflict set 3200 may initially be set empty, but entries are then stored in it so as to maintain an array of addresses known to conflict in the cache 5000. After flushing the entire cache (for example, using the x86 WBINVD instruction), lines within the set of potentially-conflicting input addresses are accessed one-by-one, until a hardware cache performance counter 1200 programmed to monitor evictions, checked after each access, detects that a first eviction has occurred. As just one example, on the Intel Sandy Bridge x86 processor, one prototype programmed the uncore CBO performance counters to monitor the LLC_VICTIMS event filtered by the MES cache states.
As described above, each address that leads to a cache eviction is added to the conflict set 3200. The cache 5000 is again flushed, and reads are performed to all addresses in the current conflict set in order to ensure that they are resident in the cache. As a result, a different conflicting input address associated with the same set will cause the next eviction. In essence, this process rotates through a ring of conflicting lines that exceed the hardware cache associativity. The entire set of potentially conflicting addresses is again read one-by-one, in the same order, and the address causing the next eviction is added to the conflict set. This process is repeated until all addresses in the set have been identified.
Note that each partition generated by this procedure will contain N+1 physical lines for an N-way set-associative cache. For example, one prototype generated partitions containing 21 physical line addresses for the 20-way set-associative Intel Sandy Bridge LLC. This also provides an alternative mechanism that can be used for computing associativity dynamically.
For large collections of input addresses, such as a region that exceeds the LLC size significantly (for example, a 100 MB region on Intel Sandy Bridge), each partition may contain more than N+1 physical lines. Various approaches can then be used to identify the remaining input memory addresses that belong to the same conflict set. For example, to find the (N+2)nd conflicting line, the cache partitioner may select any address A from among the N+1 addresses in the existing conflict set, and add it to an auxiliary “overflow” data structure 3210 associated with the conflict set 3200. The system may then remove A from the existing conflict set, so that it contains only N addresses, and also remove A from the set of input memory addresses. The conflict-set identification procedure is then continued, as before. The next address added to the conflict set will be the (N+2)nd conflicting address—counting (N+1) addresses in the current conflict set 3200, plus one address in the auxiliary overflow set 3210. This process can be repeated until all input memory addresses have been associated with partitions, with each partition consisting of the addresses in the conflict set together with its associated overflow conflict set.
After each conflict set is identified, meta-data associated with each of its constituent line addresses may be updated to mark them (
In some processors, the effective associativity of each cache set may vary across different sets. For example, using one prototype, the inventors discovered that some cache sets on the Intel Sandy Bridge LLC appeared to have only 19-way set associativity, instead of the expected 20 ways. One possible explanation for this is that it may have been due to way-partitioning performed in hardware. Some hardware units, such as the SNB integrated graphics controller, are allowed to claim a dedicated portion of the cache for their exclusive use, implemented in hardware by reserving one way of associativity from each of many cache sets. As a result, this portion of the cache is not available for use by software. The cache-set identification procedure disclosed here still computes effective set associativity information accurately in such a case. Such information is especially valuable for processors, moreover, that may employ “way partitioning” to dedicate portions of the cache for use by hardware or software to support cache quality-of-service (CQoS) features.
Optionally, the robustness and accuracy of the method described above can be further improved by reducing other sources of potential noise in the system that may generate uncontrolled cache evictions. Preferentially, the system may be booted as a uniprocessor, in order to prevent memory accesses from other processor cores sharing the LLC. Note that this is not a limitation, however, but rather an optimization—one prototype was still able to generate conflict sets successfully when the system was booted as a multiprocessor.
Similarly, it will typically be advantageous to disable interrupts while a single conflict set is being computed. On some systems, system management interrupts cannot be disabled, but can be counted; if any occurred while computing a conflict set, the computation can be repeated.
To prevent the measurement process itself from causing unwanted evictions that could affect partitioning accuracy, the code and data associated with the conflict-set computation may be mapped uncached. The system's primary data structures may then be allocated in uncached memory for this reason. Alternatively, the addresses used for this code and data can be changed dynamically, for example by maintaining multiple copies, and switching between them to prevent persistent conflicts.
More generally, the entire conflict-set computation can be repeated multiple times to ensure consistent results. Any sets that are not identical across runs, due possibly to transient effects such as uncontrolled cache evictions, can then be recomputed. In practice, the inventors have observed very few differences between the partitions generated separately by different runs; for example, it was observed that there was only a single-line difference in less than 0.2% of the partitions computed for a 20 MB cache-sized region, even without using several of the noise-reduction techniques described above.
The conflict-set computation is sufficiently performant to generate partitions dynamically at runtime for many purposes. However, for current processors, the procedure typically needs to be run only once per processor model and stepping, since the address-to-set mapping is static and identical for all instances of the same hardware.
As described above, one embodiment relies on a hardware performance counter 1200 to detect cache evictions, while accessing each line within the set of potentially-conflicting input addresses, one-by-one. While recent Intel x86 processors provide hardware counters that can be configured to count evictions, some other processors may lack this capability. In such cases, alternative implementation techniques can be used to detect evictions. Nearly all modern processors provide a hardware performance counter that can be configured to count cache misses. A cache miss counter can thus be leveraged to count evictions, since an access to a line that has been evicted will generate a miss. To determine if a cache eviction has occurred as the result of accessing a particular input address A, all input addresses accessed prior to A are re-accessed, one-by-one, in the same order, checking the hardware cache miss counter after each access. If a miss is detected, then the access to A caused an eviction. A similar method can be used on a processor that lacks even a cache miss counter, by instead timing each access (for example using the x86 RDTSC instruction); the latency of a cache miss is typically one or two order of magnitude slower than a cache hit. However, note that these alternative techniques are less efficient than using an explicit eviction counter, and may need to be repeated more times to ensure consistent results.
The method described above partitions memory units into non-conflicting sets at cache-line granularity (for example 64 bytes on x86 processors), revealing the details of how the hardware maps physical memory addresses to processor cache sets. This fine-grained information can be used directly to determine if larger memory units can conflict in the cache, by checking if any of their constituent lines can conflict. For example, since pages (for example 4 KB on x86 processors) are typically used as the unit for address translation and memory management by both processor hardware and system software, it is valuable to partition pages into non-conflicting sets.
For example, one prototype implementation for the Intel Sandy Bridge (SNB) x86 processor displayed facts about its LLC organization that could be leveraged by system software to perform page-level partitioning: A 20 MB SNB LLC contains 320K 64-byte lines. These lines are grouped into 16K 20-way-associative cache sets. SNB LLC sets are partitioned across eight hardware cache “slices”, such that each slice contains 2K sets. This means that an intra-slice cache set index can be represented in 11 bits. As noted above, the SNB LLC conflict-set data revealed that the low-order 17 address bits are identical within a single conflict set. These 17 bits encode an 11-bit intra-slice cache index (bits 6 . . . 16), plus a 6-bit intra-line byte offset (bits 0 . . . 5). This means that at 4 KB-page granularity, there will be 32 cache partitions based on address bits 12 . . . 16 (bits 0 . . . 11 are the intra-page byte offset), each with size 640 KB. This information alone yields a mechanism for performing coarse cache partitioning.
While the intra-slice cache index can be extracted directly from a physical address, each slice has a cache set at that index. The SNB LLC conflict-set data also revealed patterns in the hardware address-to-slice mapping. Note that two pages won't conflict in the cache if their constituent lines are mapped to different slices; i.e., for all lines L=0 . . . 63, L is mapped to different slices in each page. Observing one prototype, it was noted that that consecutive lines within a page were striped across different slices in one of eight regular patterns. Such information yields a second partitioning method that is distinct from traditional page coloring, based on classifying each page into one of eight slice patterns. Pages with different slice patterns do not conflict in the cache, but the slice number is not encoded as a simple bit range in the physical address.
More generally, by computing conflict sets for many regions, and examining the observed patterns, it is possible to infer or at least gain some insight into the hash function used by the hardware, which enables a more compact representation compared with using a simple lookup table that encodes the raw conflict-set data for a region. This inference may be performed manually, or by using an automated approach, such as one based on machine learning algorithms.
Using both methods of SNB cache partitioning derived from the conflict-set data, it is in many cases also possible to construct “2D” nested partitions—one using address ranges, where address bits 12 . . . 16 encode the “page color”, and the other using the page's slice-pattern classification. Note that this classification can be stored compactly, consuming only three bits per 4K page using a simple lookup table. Assuming the two partitioning methods are orthogonal, each 640 KB page-color partition can be sub-partitioned into eight slice patterns, thereby yielding a smaller, more flexible 80 KB partitioning granularity.
In many implementations, the method and system described above for partitioning memory units into non-conflicting sets will be done statically, especially if the time to complete the process is anticipated to take too long for run-time partitioning to be feasible. Nonetheless, the invention is not limited to static operation: By identifying other invariants (like a low-order 17 bits-identical property mentioned above) automatically at runtime, the system may be able to speed up its performance enough to enable dynamic, run-time partitioning of memory into non-conflicting sets. Another example of a potentially exploitable invariance might be if the target processor's cache indexing function changes only during processor initialization (for example at boot-time) and the associated indexing function is a static mapping, for example, such that a single address could not be mapped to different locations in the cache.
In some cases involving more dynamic cache indexing functions, the method described above could be combined with specific access patterns, including repeated access to some lines, in order to reveal information about the dynamic function. For example, suppose the cache indexing function can map a single address to one of two sets in the cache based on the dynamic contention for each set. By manipulating the order and number of accesses to memory addresses, an extended method could generate different levels of contention in an artificial, controlled manner, in order to identify both possible cache sets associated with each memory address.
The method described above efficiently partitions memory units into non-conflicting sets. It would also be possible, however, to use the method, for purely analytical purposes, even on a single set of memory units. For example, as mentioned above, the information that embodiments compile might be used to detect cache associativity characteristics.
In the description above, it is physical memory addresses that are processed to create efficient partitioning. It would also be possible to apply the various processing steps to virtual addresses as well, or instead, of physical addresses, assuming the mapping of virtual to physical addresses remains unchanged.
Any of the steps, operations, or processes described herein may be performed or implemented with one or more hardware or software modules, alone or in combination with other devices. In various embodiments, the cache partitioner 3000 is a software module implemented with a computer program product comprising a computer-readable medium containing computer program code, which can be executed by the CPU 1000 for performing any or all of the steps, operations, or processes described. It would in some cases also be possible to implement one or more of the embodiments using an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, and/or it may comprise a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a tangible computer readable storage medium or any type of media suitable for storing electronic instructions, and coupled to a computer system bus.
This application is a continuation of U.S. patent application Ser. No. 14/479,239 filed Sep. 5, 2014, now U.S. Pat. No. 9,477,603, which claims priority of U.S. Provisional Patent Application No. 61/874,350, filed Sep. 5, 2013.
Number | Name | Date | Kind |
---|---|---|---|
5875472 | Bauman et al. | Feb 1999 | A |
6026475 | Woodman et al. | Feb 2000 | A |
6044478 | Green et al. | Mar 2000 | A |
6129458 | Waters et al. | Oct 2000 | A |
6223256 | Gaither et al. | Apr 2001 | B1 |
6389442 | Yin et al. | May 2002 | B1 |
6697927 | Bonola et al. | Feb 2004 | B2 |
6957304 | Wilkerson et al. | Oct 2005 | B2 |
6970960 | Sarfati et al. | Nov 2005 | B1 |
7266661 | Walmsley et al. | Sep 2007 | B2 |
7434000 | Barreh et al. | Oct 2008 | B1 |
7577851 | Inamura et al. | Aug 2009 | B2 |
7657756 | Hall et al. | Feb 2010 | B2 |
7671864 | Román et al. | Mar 2010 | B2 |
7774622 | Mitra et al. | Aug 2010 | B2 |
8037250 | Barreh et al. | Oct 2011 | B1 |
8135962 | Strongin et al. | Mar 2012 | B2 |
8266676 | Hardjono et al. | Sep 2012 | B2 |
8352718 | Rao et al. | Jan 2013 | B1 |
8549288 | Bade et al. | Oct 2013 | B2 |
8615665 | Fitton et al. | Dec 2013 | B2 |
8726364 | Smith et al. | May 2014 | B2 |
8738932 | Lee et al. | May 2014 | B2 |
8782433 | Kaabouch et al. | Jul 2014 | B2 |
8812796 | Gray et al. | Aug 2014 | B2 |
8886959 | Tamiya et al. | Nov 2014 | B2 |
8904477 | Walker et al. | Dec 2014 | B2 |
8924743 | Wolfe et al. | Dec 2014 | B2 |
8949797 | Christodorescu et al. | Feb 2015 | B2 |
8990582 | McGrew et al. | Mar 2015 | B2 |
9164924 | Horowitz et al. | Oct 2015 | B2 |
9361449 | Sugano | Jun 2016 | B2 |
9477603 | Waldspurger et al. | Oct 2016 | B2 |
9639482 | Weis et al. | May 2017 | B2 |
9734092 | Weis et al. | Aug 2017 | B2 |
9747450 | Horovitz et al. | Aug 2017 | B2 |
20020004860 | Roman et al. | Jan 2002 | A1 |
20020116584 | Wilkerson et al. | Aug 2002 | A1 |
20020116595 | Morton et al. | Aug 2002 | A1 |
20020138700 | Holmberg et al. | Sep 2002 | A1 |
20030033480 | Jeremiassen et al. | Feb 2003 | A1 |
20030065892 | Bonola et al. | Apr 2003 | A1 |
20030188178 | Strongin et al. | Oct 2003 | A1 |
20030236947 | Yamazaki et al. | Dec 2003 | A1 |
20040111639 | Schwartz et al. | Jun 2004 | A1 |
20060015748 | Goto et al. | Jan 2006 | A1 |
20060020941 | Inamura et al. | Jan 2006 | A1 |
20060080553 | Hall et al. | Apr 2006 | A1 |
20060179228 | Thompson et al. | Aug 2006 | A1 |
20070239938 | Pong et al. | Oct 2007 | A1 |
20070288228 | Taillefer et al. | Dec 2007 | A1 |
20080010413 | Kailas et al. | Jan 2008 | A1 |
20080022160 | Chakraborty et al. | Jan 2008 | A1 |
20080109660 | Mitra et al. | May 2008 | A1 |
20080229118 | Kasako et al. | Sep 2008 | A1 |
20080235804 | Bade et al. | Sep 2008 | A1 |
20090094601 | Vstovskiy et al. | Apr 2009 | A1 |
20090254895 | Chen et al. | Oct 2009 | A1 |
20090328195 | Smith et al. | Dec 2009 | A1 |
20100005300 | Klotsche et al. | Jan 2010 | A1 |
20100062844 | Crowder, Jr. et al. | Mar 2010 | A1 |
20100064144 | Kaabouch et al. | Mar 2010 | A1 |
20100115620 | Alme et al. | May 2010 | A1 |
20100268692 | Resch et al. | Oct 2010 | A1 |
20100281223 | Wolfe et al. | Nov 2010 | A1 |
20100281273 | Lee et al. | Nov 2010 | A1 |
20100287385 | Conte et al. | Nov 2010 | A1 |
20110022818 | Kegel et al. | Jan 2011 | A1 |
20110040940 | Wells et al. | Feb 2011 | A1 |
20110047362 | Eichenberger et al. | Feb 2011 | A1 |
20110113260 | Ma et al. | May 2011 | A1 |
20110167278 | Goto et al. | Jul 2011 | A1 |
20110258610 | Aaraj et al. | Oct 2011 | A1 |
20110314468 | Zhou et al. | Dec 2011 | A1 |
20120124296 | Bryant et al. | May 2012 | A1 |
20120317569 | Payne, Jr. et al. | Dec 2012 | A1 |
20130067245 | Horovitz et al. | Mar 2013 | A1 |
20130090091 | Weng et al. | Apr 2013 | A1 |
20130125244 | Sugano et al. | May 2013 | A1 |
20130159726 | McKeen et al. | Jun 2013 | A1 |
20130191651 | Muff et al. | Jul 2013 | A1 |
20130254494 | Oxford et al. | Sep 2013 | A1 |
20130263121 | Franke et al. | Oct 2013 | A1 |
20140007087 | Scott-Nash et al. | Jan 2014 | A1 |
20140108649 | Barton et al. | Apr 2014 | A1 |
20140173275 | Johnson et al. | Jun 2014 | A1 |
20140201452 | Meredith et al. | Jul 2014 | A1 |
20150067265 | Weis et al. | Mar 2015 | A1 |
20150089152 | Busaba | Mar 2015 | A1 |
20150089153 | Busaba | Mar 2015 | A1 |
20150089154 | Busaba | Mar 2015 | A1 |
20150089155 | Busaba | Mar 2015 | A1 |
20150089159 | Busaba | Mar 2015 | A1 |
20150089502 | Weis et al. | Mar 2015 | A1 |
20150134932 | McNairy et al. | May 2015 | A1 |
20150149732 | Kiperberg et al. | May 2015 | A1 |
20150186295 | Long et al. | Jul 2015 | A1 |
20150227744 | Horowitz et al. | Aug 2015 | A1 |
20150269091 | Horowitz et al. | Sep 2015 | A1 |
20150378731 | Lai | Dec 2015 | A1 |
20160224475 | Horovitz et al. | Aug 2016 | A1 |
Entry |
---|
Advisory Action dated Aug. 8, 2017 for U.S. Appl. No. 14/504,203 by Horovitz, O. et al. filed Oct. 1, 2014. |
Notice of Allowance dated Jun. 15, 2017 of U.S. Appl. No. 14/663,217 by Horovitz, O., et al., filed Mar. 19, 2015. |
Non-Final Office Action dated Sep. 11, 2017 for U.S. Appl. No. 14/504,203 by Horovitz, O. et al. filed Oct. 1, 2014. |
Final Office Action dated May 18, 2017 for U.S. Appl. No. 14/504,203 of Horovitz, O. et al., filed Oct. 1, 2014. |
Non-Final Office Action dated May 26, 2017, for U.S. Appl. No. 14/497,111 of Horovitz, O. et al., filed Sep. 25, 2014. |
Notice of Allowance dated May 4, 2017, for U.S. Appl. No. 14/618,099 of Horovitz, O. et al., filed Feb. 10, 2015. |
Final Office Action dated Nov. 15, 2017, for U.S. Appl. No. 14/497,111 of Horovitz, O. et al., filed Sep. 25, 2014. |
Advisory Action dated Aug. 19, 2014, for U.S. Appl. No. 13/614,935 of Horovitz, O. et al., filed Sep. 13, 2015. |
Bugnion, E. et al., 1996, “Compiler-directed page coloring for multiprocessors”, Proceedings of the Seventh International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS VII), ACM, New York, NY, USA, 12 pages. |
Cache management via page coloring, Wikipedia, [retrieved on Nov. 12, 2015], Retrieved from the Internet: <http://en.Wikipedia.org/wiki/cache_coloring>, Nov. 6, 2015, 2 pages. |
Extended European Search Report dated Aug. 5, 2015, for European Patent Application No. 12831564.5, 7 pages. |
Final Office Action dated Jun. 5, 2014, for U.S. Appl. No. 13/614,935 of Horovitz, O., filed Sep. 13, 2012. |
First Office Action dated Aug. 2, 2016, for Japanese Patent Application No. 2014-530797, 7 pages. |
Hardware security module, Wikipedia, [retrieved on Nov. 12, 2015], Retrieved from the Internet: <http://en.wikipedia.org/wiki/hardware_security_module>, Oct. 21, 2015, 5 pages. |
International Conference on Technologies for Homeland Security (HST 2010), Nov. 2010, pp. 1-7. |
International Search Report and Written Opinion of International Application No. PCT/US12/55210, dated Jan. 25, 2013, 11 pages. |
McKeen, F., et al., “Innovative Instructions and Software Model for Isolated Execution”, Proceedings of the Second International Workshop on Hardware and Architectural Support for Security and Privacy (HASP '13), Tel-Aviv, Israel, Jun. 2013. |
Muller, T. et al., “TRESOR Runs Encryption Securely Outside RAM”, in Proceedings of the 20th USENIX Security Symposium, San Francisco, California, Aug. 2011. |
Non-Final Office Action dated Feb. 19, 2015, for U.S. Appl. No. 13/614,935 of Horovitz, O., filed Sep. 13, 2012. |
Non-Final Office Action dated Jul. 27, 2016, for U.S. Appl. No. 14/663,217 of Horovitz, C., et al. filed Mar. 15, 2015. |
Non-Final Office Action dated Jul. 28, 2016, for U.S. Appl. No. 14/497,111 of Horovitz, O., et al., filed Sep. 25, 2014. |
Non-Final Office Action dated Nov. 15, 2016, for U.S. Appl. No. 14/618,099 of Horovitz, O. et al., filed Feb. 10, 2015. |
Non-Final Office Action dated Nov. 18, 2013, for U.S. Appl. No. 13/614,935 of Horovitz, O., filed Sep. 13, 2012. |
Non-Final Office Action dated Oct. 6, 2016, U.S. Appl. No. 14/820,428 of Horovitz, O. filed Aug. 6, 2015. |
Notice of Allowance dated Jul. 15, 2015, for U.S. Appl. No. 13/614,935 of Horovitz, O., filed Sep. 13, 2012. |
Notice of Allowance dated Jun. 27, 2016, for U.S. Appl. No. 14/479,239 of Waldspurger, C. et al., filed Sep. 5, 2014. |
Peterson, Peter A.H., “Cryptkeeper: Improving Security with Encrypted RAM”, in IEEE International Conference on Technologies for Homeland Security (HST 2010), Nov. 2010, pp. 1-7. |
Ports, Dan R.K. et al., “Towards application security on untrusted operating systems”, Proceedings of the Third Conference on Hot Topics in Security (HOTSEC '08), San Jose, CA, Jul. 2008, 7 pages. |
Restriction Requirement dated Aug. 27, 2013, for U.S. Appl. No. 13/614,935 of Horovitz, O. et al., filed Sep. 13, 2015. |
U.S. Appl. No. 13/614,935, of Horovitz, O., et al. filed Sep. 13, 2012. |
U.S. Appl. No. 14/479,239 of Horovitz, O. et al., filed Aug. 5, 2014. |
U.S. Appl. No. 14/497,111 of Horovitz, O. et al., filed Sep. 25, 2014. |
U.S. Appl. No. 14/504,203 of Horovitz, O. et al., filed Oct. 1, 2014. |
U.S. Appl. No. 14/618,099 of Horovitz, O. et al., filed Feb. 10, 2015. |
U.S. Appl. No. 14/663,217 of Horovitz, O. et al., filed Mar. 19, 2015. |
U.S. Appl. No. 14/820,428 of Horovitz, O. et al., filed Aug. 6, 2015. |
Intel 64 and IA-32 Architectures Software Developer's Manual, vol. 3 (3A, 3B & 3C): System Programming Guide, Intel Corporation, Jun. 2015pp. 1-1638. |
Intel® Trusted Execution Technology (Intel® TXT), Software Development Guide, Measured Launch Environment Developer's Guide, Revision 012, Document: 315168-012, [retrieved on Nov. 12, 2015 -11-12], Retrieved from the internet: <http://download.Intel.com/technology/security/downloads/315168.pdf> Jul. 2015pp. 1-169. |
Trusted Computing Group, [retrieved on Nov. 12, 2015], Retrieved from the internet: <http://www.trustedcomputinggroup.org>, 2015, 1 page. |
Anati, Ittai et al., “Innovative Technology for CPU Based Attestation and Sealing”, Proceedings of the Second International Workshop on Hardware and Architectural Support for Security and Privacy (HASP '13), Tel-Aviv, Israel, Jun. 2013pp. 1-7. |
Baumann, A et al., “Shielding Applications from an Untrusted Cloud wth Haven”, Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation, Oct. 6-8, 2014pp. 267-283. |
Bellard, F. , “QEMU, a fast and portable dynamic translator”, Proceedings of the Use NIX 2005 Annual Technical Conference, FREEN/X Track, Apr. 2005.pp. 41-46. |
Bochs, “The Cross-Platform IA-32 Emulator”, [retrieved on Aug. 26, 2015] Retrieved from the Internet: <http://bochs.sourceforge.net/>, May 3, 20152 pages. |
Chen, X. et al., “Operating System Controlled Processor-Memory Bus Encryption”, in Proceedings of Design, Automation and Test in Europe, (Date'08), IEEE, Mar. 2008pp. 1154-1159. |
Chen, X. et al., “Overshadow: A Virtualization-Based Approach to Retrofitting Protection in Commodity Operating Systems”, Proceedings of the Thirteenth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '08) Mar. 1-5, 2008pp. 2-13. |
Deayton, Peter et al., “Set Utilization Based Dynamic Shared Cache Partitioning”, Parallel and Distributed Systems (ICPADS), 2011 IEEE 17th International Conference, Dec. 7-9, 2011, pp. 284-291. |
Iyer, Ravi , “COoS: A Framework for Enabling OoS in Shared Caches of CMP Platforms”, in Proceedings of the 18th Annual International Conference on Supercomputing (ICS '04) ACM, Jun. 26-Jul. 1, 2004pp. 257-266. |
Li, Zhiyuan , “Reducing Cache Conflicts by Partitioning and Privatizing Shared Arrays”, Parallel Architectures and Compilation Techniques Proceedings, International Conference on 1999, 1999, pp. 183-190. |
Rajimwale, Abhishek et al., “Coerced Cache Eviction and Discreet Mode Journaling: Dealing with Misbehaving Disks”, IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN), Jun. 27-30, 2011, pp. 518-529. |
Rosenblum, M. et al., “Using the SimOS machine simulator to study complex computer systems”, ACM Transactions on Modeling and Computer Simulation, vol. 7, Issue 1, Jan. 1997pp. 78-103. |
Vasudevan, A et al., “CARMA: A Hardware Tamper-Resistant Isolated Execution Environment on Commodity x86 Platforms”, in Proceedings of the ACM Symposium on Information,Computer and Communications Security (ASIACCS 2012), May 20125 pages. |
Zhang, X. et al., “Towards practical page coloring-based multicore cache management”, Proceedings of the 4th ACM European Conference on Computer Systems (EuroSys '09), ACM, Apr. 1-3, 2009, pp. 89-102. |
Non-Final Office Action dated Nov. 30, 2016, for U.S. Appl. No. 14/504,203 of Horovitz, O. et al., filed Oct. 1, 2014. |
Final Office Action dated Dec. 27, 2016, for U.S. Appl. No. 14/497,111 of Horovitz, O. et al., filed Sep. 25, 2014. |
Final Office Action dated Jan. 13, 2017, for U.S. Appl. No. 14/663,217 of Horovitz, O. et al., filed Mar. 19, 2015. |
Second Office Action dated Dec. 6, 2016 for Japanese Patent Application No. 2014-530797, 4 pages. |
Notice of Allowance dated Jan. 25, 2017 for U.S. Appl. No. 14/820,428 of Horovitz, O. et al., filed Aug. 6, 2015. |
Number | Date | Country | |
---|---|---|---|
20170206167 A1 | Jul 2017 | US |
Number | Date | Country | |
---|---|---|---|
61874350 | Sep 2013 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 14479239 | Sep 2014 | US |
Child | 15274981 | US |