Patent Grant
10102380
Embodiments of the invention relate generally to the field of information processing and more specifically, to the field of security in computing systems and microprocessors.
Securing execution and integrity of applications and their data within a computer system is of growing importance. Some prior art security techniques fail to adequately secure applications and data in a flexible but reliable manner.
Embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:
Embodiments of the invention pertain to a technique for providing secure application and data in a flexible but reliable manner. Although there are multiple embodiments of multiple aspects of the invention, the attached document entitled “Secure Enclaves Architecture” is hereby incorporated by reference as an example of at least one embodiment. However, the incorporated reference is not intended to limit the scope of embodiments of the invention in any way and other embodiments may be used while remaining within the spirit and scope of the invention.
In addition to the FSB computer system illustrated in
One or more aspects of at least one embodiment may be implemented by representative data stored on a machine-readable medium which represents various logic within the processor, which when read by a machine causes the machine to fabricate logic to perform the techniques described herein. Such representations, known as “IP cores” may be stored on a tangible, machine readable medium (“tape”) and supplied to various customers or manufacturing facilities to load into the fabrication machines that actually make the logic or processor.
Thus, a method and apparatus for directing micro-architectural memory region accesses has been described. It is to be understood that the above description is intended to be illustrative and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reading and understanding the above description. The scope of the invention may, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
Secure Enclaves is a set of instructions that provides a safe place for an application to execute code and store data inside in the context of an OS process. An application that executes in this environment is called an enclave. Enclaves are executed from the Enclave Page Cache (EPC). The enclave pages are loaded into EPC by an OS. Whenever a page of an enclave is removed from the EPC, cryptographic protections are used to protect the confidentiality of the enclave and to detect tampering when the enclave is loaded back into the EPC. Inside the EPC, enclave data is protected using access control mechanisms provided by the processor. Table 2-1 below provides a complete list of the non privileged enclave instructions.
These instructions will only execute in ring 3. All other times they will generate a #UD fault. Table 2-2 provides the list of privileged instructions.
Enclave Page Cache (EPC) is where enclave code is executed and protected enclave data is accessed. The EPC is located within the physical address space of a platform but can be accessed only using SE instructions. The EPC may contain pages from many different enclaves and provides access control mechanism to protect the integrity and confidentiality of the pages. The page cache maintains a coherency protocol similar to the one used for coherent physical memory in the platform.
The EPC can be instantiated in several ways. It could be constructed of dedicated SRAM on the processor package. The preferred implementation mechanism is known as Crypto Memory Aperture. This mechanism allows the EPC to be large. More details of CMA are in the section below.
The Enclave Page Cache Map (EPCM) contains the state information associated with each page in the EPC. This state provides the information such as the enclave that the page belongs to, the state of a loaded page, etc. When a page is removed from the EPC, the state information is exported as well and is protected using cryptographic means. When an enclave page is re-loaded into the EPC, the state information is verified.
CMA is a mechanism which provides support for instantiating the EPC, EPCM, and other SE related structures. The aperture is a region of the physical address space which is reserved for this use.
The EPC and EPCM (as well as other implementation data structures) are mapped to a location inside the aperture. The backing store is the actual data for these resources. When a memory request for the EPC is generated CMA remaps to the backing store location containing the encrypted EPC data and retrieves the data.
In general most of SE is implemented in microcode. There is hardware support required in several places including CMA, logic controlling data movement outside the package, and in the cores.
Any page faults or exceptions which occur while executing inside of the enclave are reflected by the microcode to the responsible OS or VMM. When the machine is not executing inside an enclave, access control to the EPC is provided by the SE range register (SERR). When the machine is running inside the microcode provides page table level protections which prevent access to other EPC entries not belonging to the executing enclave.
One option to implement secure enclaves is to implement the instructions and the protections using the microcode capability in some processors. This capability may meet the security requirements that secure enclaves requires to meet its goals.
The SERR register as shown in
One option in the implementation to improve performance is to provide a bit or a few bits to indicate entries in the Translation Lookaside Buffer (TLB) are for an enclave or a particular enclave. If these bits are not provided a TLB flush will be needed when exiting the enclave to prevent other code from accessing the enclave.
The enclave bit is compared to the enclave mode bit. Extra bits would provide an enclave space id functionality. A particular enclave would be assigned an id. The id would be compared with the id of the executing enclave as part of the address check. TLB support is an optional performance enhancement. When an entry may be invalidated in the TLB due to the removal of EPC data, a special microcoded shootdown mechanism is needed. In one embodiment microcode may contact all other cores in the enclave trust boundary and verify the entry is no longer in any TLB. Other embodiments may provide a means for microcode to be assured that other processors have invalidated the TLB entries.
To prevent DMA snoops and invalidates to the EPC a special SAD and/or TAD entry is provided. These dedicated registers provide the protection of the EPC. This is set to the same values as the SERR.
In order to ensure secure keys for each enclave Secure Enclave microcode may use secure access to random numbers in one embodiment.
An enclave may be protected against tampering. The details of the mechanism used for tampering protection vary by implementation. When an enclave is tampered it will prevent further execution on the thread which detected the tampering. In order for users to understand the state of an enclave there is an attestation mechanism put into place to provide proof of the enclave build. This includes the EREPORT instruction used to present information on the enclave contents.
In order to simplify the microcode code required in the enclave design the concept of architectural enclaves was developed. These enclaves are given special access privileges based on the original of the code for the enclave.
The enclave state across power cycles is dependent on software policy. Data inside the CMA is lost on power down. Software may ensure that the enclave data is not lost on a power cycle if it would like to preserve the enclave. Data resident in the EPC may be flushed to memory if software wishes to keep enclaves alive across S3 power states. Software could elect to require that applications tear down all enclaves when power is removed.
An enclave is protected differently depending on its location. Data external to the CPU package is protected using encryption and integrity checking. For code and data in the enclave page cache, pages are protected using access control mechanisms.
Enclave pages residing in system memory are protected using encryption and integrity. During the load of the page into the EPC, the page is copied into the EPC, decrypted and page's integrity is checked.
When an enclave page residing inside the EPC is stored to system memory, it is encrypted with the enclave key. Authentication information is also stored at the time of the page store. Enclave data inside the EPC is unencrypted and protected by access control mechanisms. The processor protects this data so that only the enclave which owns that data can access it.
When enclave pages residing in the EPC is evicted from the cache to main memory outside the CPU package, it is protected by CMA encryption. The CMA will encrypt the data to provide data confidentiality. The integrity of the EPC is provided by range registers that prevent reads and writes to the EPC
When an application desires to load an enclave it will call a system routine in the OS. The OS will attempt to allocate some pages in the EPC. If there is no open spot then the OS will select a victim enclave to remove. The OS will evict the pages of the victim enclave using the EWBINVPG instruction for each page. When the OS has completed the eviction, it will add the secure enclaves control structure (SECS) to the enclave using the ECREATE command. After the SECS is created, the OS will add pages to the enclave as requested by the application using the EADDPRE instruction.
To add data pages to the enclave, the OS may first add SMAP pages to the enclave using the EADDSMAP instruction. Depending on the size and layout of the enclave the OS will add several SMAP pages. When all of the enclave pages are added to the enclave the OS will execute the EINIT instruction to enable the enclave to be executed. A parameter to the EINIT instruction is a permit which demonstrates that the enclave is licensed to run on that machine. When an application is loaded a permit needs to be created. After EINIT successfully completes, the application can execute the EENTER instruction to enter the enclave.
When an enclave is built and marked for execution the application may need to add or subtract physical memory inside the enclave. To support this there are instructions which allow additional memory to be added to the enclave. To add memory to the enclave, the memory is allocated to the correct linear address inside the enclave. The OS copies this memory page into the EPC indicating the linear address. The EADDPOST instruction is run to add this memory to the enclave. If the SMAP node is not resident inside the EPC it may be loaded first.
After the memory is copied the enclave software may accept the page before it can be accessed inside. The enclave accepts the data by executing the EACCEPT instruction. This instruction can only be executed by the software inside the enclave.
In some cases the software may want to modify the properties of the enclave memory. In order to do the change the SMAP may be updated. For instance the software may want to create another thread entry point, TCS inside the enclave. In this case the enclave requests that the OS change the SMAP properties of the page using the EMODIFY instruction. After the properties are changed, the enclave software executes the EACCEPT instruction to allow the page to be used.
Memory pages can be removed from the enclave. When the enclave is ready to remove a page, it sends a request to the OS. The OS will execute the EREMOVE instruction which will remove the page from the SMAP. The EREMOVE instruction also invalidates the EPC entry.
To ensure the integrity of the enclave environment a number of access checks may be done. Among the various security properties enforced is that data is correctly located in the EPC to prevent data from leaking across enclaves and the referencing address is not corrupted to assure that code is not moved to a different linear address in the enclave.
The access protection requirements can be implemented using a ranger register and microcode managed shadow page tables. In another embodiment, to avoid the overhead of shadow page tables, the page miss handler hardware can be modified to perform the same access control requirements.
The EPC is accessible to the logical processor (LP) only if the LP is either executing in microcode mode, or if the LP is executing inside an enclave and the linear address being accessed belongs to the linear address range covered by that enclave. In other words, only microcode accesses or enclave accesses are allowed to go to the EPC range. Any other accesses to the EPC range are considered illegal.
An enclave access may be resolved to a physical address belonging to the EPC. If the access falls outside the EPC but the linear address indicates the address is inside the enclave then the access may be stopped. A fault to the OS or the instruction is reported.
The access to an address in the enclave may be located inside the EPC for the access to succeed. The check that the entry is present in the EPC is usually done by checking the EPCM to verify the valid bit. Each EPC page is dedicated to a particular enclave. References to that EPC entry can only be made by the enclave who owns the EPC page. This is checked by validating the referenced page matches the SECS of the executing enclave.
Each EPC page represents a particular linear address page for the enclave. The requested linear address may match the linear address of the page in the EPC. For instance the EPCM entry stores the linear address for which an enclave page was brought into the EPC. When an enclave access resolves to an EPC page, the linear address for which the page was brought in may match the linear address of the current request.
The linear address mapping of an enclave cannot be corrupted. If the page tables of the linear address are corrupted the resulting access is illegal. This prevents an attacker from moving code and data around inside the enclave.
When the OS/VMM adds a page to an enclave after it has been initialized, the EADDPOST instruction sets the “pending” bit in the EPCM for that page. The pending bit survives subsequent EPC write-backs and evictions (using SEC_INFO). The enclave may issue EACCEPT to clear the pending bit. If an enclave access resolves to an EPC page for which the pending bit is set, the LP issues EF_PENDING fault to the OS/VMM.
When the OS/VMM loads a replay-protected enclave page to the EPC, it sets the FCR (Freshness Check Required) bit in the EPCM entry for that page. The OS/VMM can clear this bit by executing EUPSMAP instruction on that EPC page to clear this bit. An enclave access is allowed to continue only if the FCR bit on that page is not set. Otherwise, the LP delivers EF_FRESH_CHK fault to the OS/VMM.
Each EPCM entry contains a “dirty” bit which indicates whether an enclave is allowed to write to that page. An enclave is allowed to write to an enclave page only if the dirty bit for that page in the EPCM is set. If such is not the case, the LP issues EF_EWRITE to the OS/VMM. The OS/VMM can set the dirty bit by executing the EUPSMAP instruction on that page.
Any time a logical processor is executing inside an enclave, that enclave's SECS pages may be present in the EPC. However, the SE security model requires that an enclave may not be allowed to make any direct memory accesses to its own SECS (otherwise the enclave will be able to read its own enclave key, completely compromising the security). If an enclave access resolves to an EPC page that holds the SECS for that enclave, the OS/VMM is notified via EF_ATTRIB_SECS fault. An enclave is not allowed to modify any pages that have a TCS attribute set. If an enclave attempts to modify a TCS loaded into the EPC, the OS/VMM is notified via EF_ATTRIB_TCS fault.
In the Size field of the tables below, the following values and indicators are used:
Note: Some fields have names beginning with a lower case “o” (e.g. oLSP). These fields are pointers, but are represented in the enclave as offsets relative to the base of the enclave. This representation ensures that the measurement of enclave pages is independent of the location at which the enclave is created.
Note: Fields are not described in any particular order (yet). Some fields may be moved to different memory pages within their respective data structures to allow, for example, for different means of protection.
Associated with each thread is a Thread Control Structure (TCS). The TCS contains:
The thread state can have one of 5 values:
State Save Area Offset (oSSA)
The State Save Area Offset (oSSA) points to a stack of state save frames used to save the processor state on an interrupt or exception that occurs while executing in the enclave. Next State Save Area (NSSA) is used by the interrupt microcode to determine where to save the processor state on an interrupt or exception that occurs while executing in the enclave. It is an index into the array of frames addressed by oSSA. Count of Save Areas (CSSA) specify the number of SSA frames available for this TCS. When an interrupt or exception occurs and there are no more SSA frames available (NSSA>=CSSA), the interrupt or exception will still occur and the processor state will be cleared, but the TCS will be marked as INVALID.
On an interrupt occurring while running in an enclave, the machine state will be saved in the TCS::SSA (State Save Area). This area includes:
The TCS::SSA may not be paged out at the time an interrupt occurs. EENTER checks that SSA is inside the EPC and caches the physical address. In the event that the page is evicted, the processor executing the EWBINVPG will force an enclave exit on the processor currently executing the thread using the SSA and report a page fault to it.
Page Information (PAGE_INFO) is an architectural data structure that is used as parameter to the EPC-management instructions.
The SEC_INFO flags and EPC flags contain bits indicating the type of page.
The SEC_INFO Flags are a set of bits describing the state of an enclave page.
Security Information (SEC_INFO) data structure holds cryptographic meta-data that is needed for forgery protection.
Certificate (CERT) is the certificate structure provided with Architectural Enclaves and passed to EMKPERMIT. This structure is 4096 byte and may be page-aligned.
Permit (PERMIT) outputted from EMKPERMIT and the Permit Enclave and consumed by EINIT. It is 4096 bytes and may be page-aligned.
The ERPORT structure is the output of the EREPORT instruction.
Measurements (MEASUREMENTS) is the output parameter of the ERDMR instruction. It contains the Measurement Register values of an enclave, taken from a specified SECS.
Key Request (KEY_REQUEST) is an input parameter to the EGETKEY instruction. It is used for selecting the appropriate key and any additional parameters required in the derivation of that key.
This structure is used by key derivations to generate keys based on the security versions of the enclave and the enclave's SE TCB. See the Platform TCB Recovery Specification for more details on the TCB Security Version structure.
The EPCM Flags are a set of bits describing the state of an enclave page.
Enclave Page Cache Map (EPCM) is a secure structure used by the processor to track the contents of the page cache. The EPCM holds exactly one entry for each page that is currently loaded into the EPC.
Attestation is the process of demonstrating that a piece of software has been established on the platform especially to a remote entity. In the case of secure enclaves it is the mechanism by which a remote platform establishes that software is running on an authentic platform protected within an enclave prior to trusting that software with secrets and protected data. The process of attestation has three phases, Measurement, Storage and Reporting.
There are two periods of measurement inside an enclave pre-enclave establishment and post-enclave establishment. It is the responsibility of the enclave instructions to provide measurements of the enclave as it is established. Once the enclave has been established the software inside the enclave becomes responsible for measurement.
The act of reporting cryptographically binds measurements made when creating the enclave to the platform. This mechanism is often referred to as Quoting as this type of functionality has been available on the platform for sometime as a TPM command. The values of the Measurement Registers (MR) are concatenated and then signed using an asymmetric key. Any challenger simply then has to verify the signature over the quote structure in order to validate the quote.
Due to the nature of the computation involved with asymmetric keys and our desire to reduce the number of instructions in the enclave leaf we will not be including instructions to do asymmetric signing. Our approach, shown in the figure below, is to provide a hardware based mechanism for producing ‘reports’ based on a symmetric key authentication key, and to allow these symmetric key based ‘reports’ to be converted into asymmetrically signed ‘quotes’ using software which itself is protected using an enclave. As the Quoting Enclave needs to be authorized to have access to the platform attestation key the Quoting Enclave itself is a special purpose enclave, known as an Authenticated Enclave.
Each enclave provides two 256-bit wide Measurement Registers (MR_EADD & MR_POLICY) and two reserved registers. These measurement registers are contained within the SECS of the enclave.
MR_EADD contains the aggregated measurement of the enclave as it was built using the EADD instruction before the EINIT instruction is called. It is only written to by microcode and therefore it needs to be placed in a page of the SECS which is read-only by enclave code. On each invocation of EADD it computes a SHA256 over the page data and the security meta data associated with that page, namely the relative address (w.r.t. to the enclave's base address) of the page and the page's SEC_INFO.flags and this value is extended into MR_EADD 1100. Where we define ‘extend’ to mean:
New MR Value=Hash(Old MR Value∥Input Value)
MR_POLICY contains the value of the policy used to authenticate the policy which permitted the enclave to be launched. This value was taken from the enclave permit which was placed in the SECS at launch and copied as a successful completion of the EINIT instruction. MR_POLICY is only written to by microcode and therefore it needs to be placed in a page of the SECS which is read-only by enclave code.
The EREPORT instruction creates an intermediate key to perform a symmetric key based GMAC over the measurement registers, user data, and additional contextual information, such as the enclave's capabilities and flags.
In addition to the Measurement Registers the user can also supply a 256 bit wide block of data for inclusion in the report. There are many application specific values, e.g. a challenger NONCE and/or an application created key, which the user would like to attest. These values can be reduced to a single hash and submitted to the report for inclusion.
In order to prevent key wear out, by repeatedly calling EREPORT, a random 128 bit value (known as reportKeyID) is produced on each power cycle of the processor and stored in internal location. This value is incremented after 2^32 AES operations using this value. Each call to the EREPORT instruction will increment this value by 1 in one embodiment.
The Flags field in the report structure can be used to determine certain state information about the enclave or when the EREPORT instruction was called which will be useful for a challenger to assess whether they may trust the enclave.
In one embodiment the architecture allows an architectural enclave with the appropriate capability set to retrieve the key used in the CMAC operation with the EGETKEY command and hence verify that the report was created on the hardware it is currently running on. This capability is limited to the Quoting Architectural Enclave.
For retrieving measurements of the enclave when executing outside the enclave, the ERDMR (Read Measurements) instruction is provided. This instruction takes a pointer to a valid SECS page and a pointer to address where the measurements will be delivered. The measurements are delivered in the form of a MEASUREMENT structure. The MEASUREMENT structure is not cryptographically protected.
Enclave pages when not inside the Enclave Page Cache are protected cryptographically. There are three levels of cryptographic protection: Confidentiality Protection, Forgery Protection, and Replay Protection. In one embodiment application are allowed to choose a protection level for each enclave page independently of the protection level chosen for other pages of the same enclave. The enclaves' implementation MAY allow applications to choose between the following combinations: Forgery Protection, Forgery Protection and Replay Protection, Confidentiality and Forgery Protection, and Confidentiality, Forgery Protection, and Replay Protection. Confidentiality and forgery protection on enclave page can be achieved using one of the several authenticated encryption modes such as the Galois Counter Mode (GCM) in conjunction with an appropriate cipher such as AES. Replay protection, however, requires a more sophisticated solution.
Replay protection ensures that all the contents of an enclave as seen by a logical processor at any given time belong to a single snapshot of a non-corrupted enclave. Thus, a replay-protection mechanism needs to define the concept of an enclave version and provide a mechanism of determining whether a forgery-protected enclave page belongs to that version of the enclave. To this end, the replay-protection mechanism ties the contents of each forgery-protected enclave page to a page version number using a message-authentication algorithm such GMAC. In the case of GMAC, the version can be used as a part of the initialization vector (IV) as shown in
To extend versioning to the entire enclave, the replay-protection mechanism maintains a version tree. Leaf nodes contain the versions of individual replay-protected pages of the enclave instance. Each internal node provides the version of each group of children and therefore logically holds the version information for the pages they are representing.
In one embodiment, tree structure was chosen to reduce the number of data that needs to be processed from O(n) pages to O(log n). The use of a version tree instead of a hash tree was selected to allow page eviction from the EPC without necessitating a tree update.
Since replay protection may require each page to have its own version that is cryptographically tied to its contents, Replay Protection requires Forgery Protection. Thus, forgery protection in SE architecture is mandatory. Additionally, initial implementations of SE may further restrict the list of supported protection combinations.
The OS/VMM creates an enclave by executing the ECREATE instruction. During the creation of the enclave, the range of linear addresses that is protected by the enclave is specified. This range of linear addresses is known as the Enclave Linear Space (ELS) range.
Once an enclave is created, individual pages belonging to the ELS range are added to the enclave using the EADDPRE instruction. The EADDPRE instruction brings each of the added pages into enclave protection domain by moving those pages into the Enclave Page Cache. If any of these pages out of the EPC using EWBINVPG the logical processor will put cryptographic protections on those pages.
The cryptographic protections are achieved by associating cryptographic meta-data with each enclave page. This meta-data is used by the uCode flows for various processor instructions to decrypt the contents of an enclave page and to verify the authenticity/freshness of each enclave page. The SE architecture provides several such instructions to update, manage, and validate the cryptographic meta-data.
Each enclave page has Security Information SEC_INFO data structure associated with it. The purpose of the SEC_INFO data structure is to hold the cryptographic meta-data required to decrypt and verify the page. The various fields of the SEC_INFO structure are as follows.
Security Information Flags (SEC_INFO.Flags) describe the page type, cryptographic and access protection for a protected page.
Security Map (SMAP) is the data structure that is used to store cryptographic meta-data required to verify the freshness of an enclave page (i.e., replay protection). A security map represents a full version tree for a particular snapshot of an enclave. Each node of the Security Map holds versions for 256 child nodes (or enclave pages, in the case of a leaf node). Additional meta-data about the security node is contained within the SEC_INFO for a particular SMAP node.
In one embodiment, the Security Map tree is two levels deep1, and is accessed using enclave offset of an enclave page within that enclave. The root of the SMAP is contained within the SECS and it only holds versions for 128 child nodes. Bits from the enclave offset are used to choose appropriate child are used to index the SMAP. In gen 1, the enclave offset is 35 bits long. The enclave offset is extracted by the following formula (enclave linear address & enclave mask). The enclave mask is determined by (size of the enclave−1) and can be calculated during ECREATE. 1The depth of the Security Map is related to the size of the enclave supported by the SE architecture. In Gen 1, SE architecture will support maximum enclave size of 32 GB.
In general, at depth l>1 bits N−(l)×8 through N−(l+1)×8+1 are used to select the appropriate child at next level.
Note: Security Map is a logical data-structure, and is not architectural. A logical processor is not even aware of where in the linear address space the SMAP is located. The system software is responsible for maintaining and walking the security map. Each individual node in the security map has an architecturally defined structure—however, the architecture does not specify how the security map is maintained in the memory. It may however be noted that, each node in the security map has a well-defined logical position in the security map, and if the node is moved around within the map, the various processor instructions that relate to the security map will interpret that as an attack scenario.
A root security node is contained within the SECS and contains version information for 128 children. A non-root security node is protected page and its associated SEC_INFO. The protected page contains version information for 256 children.
The SEC_INFO contains the location of the SMAP within the SMAP. The location with the SMAP is determined by the linear/enclave offset and the page type SMAP_LEVEL_1 and SMAP_LEVEL_2.
Adding a replay-protected enclave page requires that the SMAP parent have been created and resident inside the EPC with FCR bit cleared. To verify the integrity of an enclave page, a logical processor uses the IV_P and key_id in the SEC_INFO structure to generate a key. The key is used to compute the MAC over the flags in the SEC_INFO structure and the contents of the page. The computed MAC is compared with MAC located in the SEC_INFO structure. If the MACs match, then the page is considered to pass the integrity check.
A logical processor verifies the integrity of a page when the page is loaded into the EPC using the ELPG instruction. As a part of this instruction, a logical processor notes down the IV_P from the SEC_INFO structure that was used to verify the page.
To verify the freshness an enclave page, a logical processor verifies that the enclave page and its smap parent have been loaded into the EPC and that smap parent is fresh. It then proceeds to check the version of the page against version of stored in the smap parent. If the two versions match, the processor generates a new version for the page and updates the version in the smap parent and version of the enclave page. Lastly, it marks the enclave page as fresh.
Note—the generation of a new version allows the page to be modifiable. This both simplified the architecture and implementation.
To remove an enclave page, a logical processor verifies that the enclave page and its smap parent have been loaded into the EPC and are both fresh. It then proceeds to set the version of the page in the smap parent to 0 and mark the EPC slot of the enclave page as available.
The Enclave Page Cache (EPC) is a secure storage used by the CPU to temporarily store enclave pages when they are not cryptographically protected by SE cryptographic protections.
Following requirements are identified on the EPC. Any accesses to the enclave memory pages loaded into the EPC that belong to non-debug en-claves may be protected from any modification by software entities outside that enclave. Attackers may not be able to read plain-text data belonging to non-debug enclaves that is loaded into the EPC via straight-forward hardware attacks. Attackers may not be able to able to modify data in the EPC that belongs to non-debug en-claves via straight-forward hardware attacks. Any data loaded into the EPC may be accessible coherently, yet securely from any CPU in the system.
There are several mechanisms of implementing the EPC. The EPC could be implemented as on on-die SRAM or eDRAM. The EPC could also be constructed by dynamically sequestering ways of the CPU's last-level cache. In such an implementation, the EPC may be protected from un-authorized accesses from outside the package. However, other packages in the system may be able to access the EPC coherently, yet securely.
Another mechanism of implementing EPC is the Crypto Memory Aperture (CMA). The Crypto Memory Aperture (CMA) provides a cost-effective mechanism of creating cryptographically protected volatile storage using platform DRAM. The CMA uses one or more strategically placed cryptographic units in the CPU uncore to provide varying levels of protection, as needed by the customer technology. The various uncore agents are modified to recognize the memory accesses going to the CMA, and to route those 25 accesses to a Crypto Controller located in the uncore. The Crypto Controller, depending on the desired protection level, generates one or more memory accesses to the platform DRAM to fetch the cipher-text. It then processes the cipher-text to generate the plain-text, and satisfies the original CMA memory request. The CMA fully integrates into the Intel QuickPath Interconnect (QPI) protocol, and scales to multi-package platforms, with security extensions to the QPI protocol. In a multi-package platform 30 configuration, the CMA protects memory transfers between Intel CPUs using a link-level security (Link-Sec) engine in the externally facing QPI link layers.
An SECS is said to be active if it is currently loaded into the EPC. As explained later in this document, the OS/VMM is responsible for managing what gets loaded into the EPC. However, while loading an enclave page into the EPC, the OS/VMM needs to tell the CPU the whereabouts of the SECS for that page, except when the page under consideration itself is an SECS. When the page being loaded is not an SECS, the CPU requires that the SECS corresponding to the page be located inside the EPC. Before loading any page for an enclave, the OS/VMM MAY load the SECS for that enclave into the EPC.
It may be noted that, the CPU does not enforce any restrictions on how many times an SECS could be loaded to the EPC—however, it would be highly unusual for the OS/VMM to load multiple copies of the SECS to the enclave page cache. Nevertheless, even if multiple copies of the same SECS are loaded to the EPC, each of those copies is considered as a separate active SECS instance, and enclave pages loaded into the EPC that belong to different instances of active SECS are considered to belong to different enclaves by the hardware.
The OS/VMM sees the EPC as a contiguous block of physical memory in the system address space. 10 However, to reduce internal storage, and enable fast indexing, the CPU associates a slot identifier (SID) with each EPC page. The physical address of an EPC page and the corresponding slot identifier are related to each other as follows.
sid=(page_pa−epc_base_pa)>>12
page_pa=pc_base_p|(sid<<12)
The hardware uses a special slot identifier of 0xFF to denote an invalid slot. EPC slot identifiers are used by both the uCode and the PMH to track the information about the enclave pages.
Every enclave page loaded to the EPC has a well-defined system physical address. Since there is a one-to-one mapping between the physical addresses belonging to EPC and the EPC slot identifiers, we say that each page loaded to EPC has its own EPC slot identifier or EPC_SID.
Additionally, every enclave page, except for the SECS page, that is loaded into the EPC is associated with an active SECS instance. Recall that an active SECS instance is nothing but an SECS page that is loaded to the EPC. Consequently, the active SECS page also has its own EPC_SID. The EPC_SID of the SECS page to which a non-SECS enclave page belongs is referred to as the SECS_SID for non-SECS 25 page. For each page loaded into the EPC, the hardware keeps track of the SECS_SID. The SECS_SID for an SECS pages loaded into the EPC is defined to be 0xFF, or the invalid SID.
The EPCM is a secure structure used by the processor to track the contents of the page cache. The 30 EPCM holds exactly one entry for each page that is currently loaded into the EPC. For the page represented by it, each EPCM entry tracks such information as the enclave to which that page belongs, the linear address for which the page was brought into the enclave page cache, the version of the page, etc. The EPCM structure is used by the CPU in the address-translation flow to enforce access-control on the enclave pages loaded into the EPC. The EPCM entries are managed by the (x)uCode as part of various instruction flows.
In one embodiment of the invention, an enclave page cache (EPC) may be dynamically allocated or de-allocated. In one embodiment, software, such as an operating system can dynamically allocate pages in memory as EPC or de-allocate memory from EPC. In one embodiment, the operating system can assign any page in the enclave to be in the EPC. The EPC can take up every available location in the memory in some embodiments. One distinction of dynamic EPC from fixed EPC, according to one embodiment, is that dynamic EPC allows for the addition and removal of pages of memory. In one embodiment, logic, such as a software driver may allocate a memory area to be EPC and de-allocate the memory from the EPC. In one embodiment, a pre-boot process checks for available memory to store meta data for each page of memory and software may declare a page to be EPC or non EPC, while hardware logic may track and enforce each page's attributes.
In one embodiment, hardware logic may control access to the memory used as an EPC via a translation lookaside buffer (TLB) and a page miss handler (PMH). In one embodiment, when the search address has a match in the TLB, known as a TLB hit, the TLB may be flushed when the secure enclave exits the EPC. In one embodiment, when the search address has no match in the TLB, known as a TLB miss, an extra lookup may fetch data from the enclave page cache map (EPCM) on multiple memory references. In one embodiment, a PMH may perform the look up of the EPCM. In another embodiment a range register in the PMH is checked to control access to a contiguous physical address, EPC. The operating system may not allow direct memory access (DMA) to access the EPC pages. If the returned page of the memory is marked as an enclave page, the secure enclave control structure identification (SECSID) of the page may be checked against that of the currently executing enclave to ensure that the access is secure. If there is a mismatch between the SECSID of the returned page and that of the currently executing enclave, the PMH may issue an abort message. If the returned page of the memory is not marked as an enclave page or if the returned page of the memory is marked as an enclave page and the SECSID of the page matches that of the executing enclave's, the PMH may load the page translation into the TLB. In one embodiment, a cache tag can be used to identify the enclave line from the other lines on a writeback cycle. However, in at least one embodiment, a cache tag is not used if the logic determining the type of memory request accesses the EPCM during a writeback cycle.
In one embodiment of the invention, software, the BIOS, can allocate memory before the operating system boots to create enclave pages. Software may, in one embodiment, create an EPC with a sequence of steps in the BIOS. The BIOS may reserve some memory to store meta data and, for each processor, set a range register. BIOS may take as input a base address and a memory size. The system configuration is checked by a process known as MCHECK to ensure all registers on all packages and all cores are set correctly to provide protection from accesses outside the enclave. MCHECK will lock the registers until the system resets. In another embodiment, software can add a page to an EPC by an instruction known as EPCADD, which declares portions of memory to be a part of the EPC. The EPCADD sequence can take a memory address as input and can output a message to indicate the success or failure. In the case of EPCADD outputting a message indicating success, EPCADD can set the EPCM.E bit and the page corresponding to that physical address is flushed from all TLBs in the system. In one embodiment of the invention, the EPCADD may return an error code in RAX of 01 to represent the page with the input address is already an EPC page and an error code of 02 to represent the input address is out of range. A page of memory declared by EPCADD as part of an EPC may require EPC semantics to access the data. In this embodiment of the invention, software can remove a page from the EPC in a instruction known as EWBINVPG and allow the encrypted data to continue to be available while protected by cryptography and integrity protection. Data in this format can be stored on regular memory of the hard disk drive. In yet another embodiment, software can, in an instruction known as EPCREMOVE, remove a page in an EPC and make the encrypted data unavailable. Hardware executing EPCREMOVE clears the page and parts of the EPCM. EPCREMOVE can be executed without first executing EWBINVPG. The EPCREMOVE sequence can, in one embodiment, remove a page from an EPC based on a memory address. In an embodiment of the invention, the EPCREMOVE instruction may contain an error code in RAX of 01 to represent that the page being removed is part of a secure enclave control structure (SECS) and cannot be removed and an error code of 02 to represent that the page being removed is not an EPC page. A global TLB shootdown of a page of memory can result from EPCREMOVE in one embodiment of the invention, and the memory formerly occupied by the page could become available for general software access.
The PMH prevents access to the protected regions of the memory space. Depending on the architecture this can be as simple as just the physical address check of accesses to the EPC. Further PMH support can be used to allow for performance improvements or alternative implementations of SE. SE architecture relies on the Page-miss Handler (PMH) to prevent unauthorized accesses to the enclave pages loaded into the enclave page cache. PMH detects various events, and reports those events back to microcode. The microcode may report an event to the OS/VMM. The OS/VMM then can execute appropriate instruction to remedy the fault.
When an enclave is created using the ECREATE instruction, a linear address range is specified for that enclave. This range is called the linear address range for that enclave. Any memory pages belonging to the linear address range of the enclave are considered to be under the enclave's protection, and have SEC_INFO entries associated with them.
Memory pages belonging to the linear address range of an enclave are also referred to as enclave pages. A program executing inside an enclave is allowed to access the enclave pages only if those pages are loaded into the enclave page cache and it is the enclave which owns the page. The processor will generate an exception-class event if this is not the case. It is the responsibility of the OS/VMM to ensure that the enclave pages get loaded to the EPC as needed.
If a logical processor is executing an enclave, and it generates a memory access to its enclave page, then such a memory access is referred to as an enclave access. The address may be checked to ensure it is being accessed by the correct entity
In one embodiment the PMH provides access control functionality to protect the EPC when a program is not executing in an enclave. A range register, enabled for each logical processor will restrict access to the EPC when the processor is not executing enclave code. This range register is disabled when the processor starts executing enclave code. In its place the processor puts special page tables in place. These page tables are controlled by the processor and only allow access to EPC pages owned by that enclave. The processor and microcode restrict access to the EPC using these two mechanisms.
In some embodiments, a tradeoffs can be made among many axis including performance, implementation complexity, and silicon cost. In this chapter three possible implementations are described such that developers can understand some of the possible tradeoffs. Table 8-1 below shows these possible protections and the PMH support required.
As shown in the first row of Table 8-1 one additional range register is all that is required to provide the access control protections needed. In this particular implementation the other protections are provided by microcode. The range register may be enabled on a logical processor basis. The basic implementation using this mechanism is shown in
PMH is modified to prune out accesses to the CMA range (covered by CMRR in the CPU) from LPs that are neither running in microcode mode nor in enclave mode. Additionally, LPs running in enclave mode are only allowed to access the EPC sub-range of the CMA.
When a page is not present in the EPC a fault is provided to the OS/VMM to indicate this fact. The Page Fault Error Code Map is altered as shown in Table 8-2. This indicate the new bits which are used to report the faulting condition. If there is no EPC fault then bit 5 is set to zero and bits 6 to 9 are also zero. If the fault is due to an EPC condition then bit 5 will be set and the software may decode bits 6 to 9 to understand the EPC faulting condition. More information on the fault types is described in the next section.
When bit 5 of the Page Fault Error Code is set bits 6 to 9 are interpreted as given in Table 8-2. This shows the condition which caused the page fault to occur. Some of the states indicate an illegal condition which may never occur in normal operation. They indicate an OS/VMM management error.
In order to protect the EPC from attack there may be a mechanism which invalidates EPC addresses in all TLB's on the platform. This feature may signal to all cores that a particular page is to be invalidated. It may then wait until all processors return an indication that the shoot down is complete.
Whenever an enclave exit, EEXIT, occurs the TLB may not allow accesses to the enclave pages currently present in the TLB. This can be done by clearing the TLB or using extra bits to tag the enclave entries.
One alternative is the use of an enclave bit in the TLB on enclave exit all the enclave entries are cleared. Another alternative is the use of several bits to indentify a particular enclave. In this case the enclave entries do not need to be evicted. The enclave entries can be left in the tlb. When an address is sent to the tlb for lookup these bits are appended to the lookup. These bits are compared to an enclave id from the core which indicates the enclave identity. If the bits match then the request came from the same enclave. If the match fails then the request did not come from that particular enclave and the lookup will not hit on that location.
Enclave Authentication provides a means of determining the authority that licensed the enclave code to run within an enclave, which is the author/approver of that code. Enclave Authentication also provides a foundation to outsource Enclave microcode flows, Flexible Sealing & Reporting, as well an enforcement point for a number of new business models.
Certain aspects of the Secure Enclaves architecture require complex, time consuming flows, which are not well suited for implementation within micro-coded instructions. The solution is to outsource those portions of the Secure Enclaves architecture to macrocode. In many cases, the outsourced code requires special access to sensitive processor or platform data. For example, EPID signing is too long for a single instruction. Instead a Quoting Enclave is used to produce EPID signed Quotes, by granting it special access to the EPID private key. Enclave authentication allows Intel to specify the additional capabilities granted to specific enclaves, such as access to the EPID key only by the Quoting Enclave. Enclaves provided by Intel, which have additional capabilities and implement core Enclave functionality, are referred to as Architectural Enclaves.
Enclave Sealed Storage provides enclave software with the ability to encrypt data to certain attributes of the enclave, such as its load-time measurement. Enclaves Attestation framework allows an enclave to provide evidence of the enclave's measurement to an external party. In many circumstances, it is more desirable to seal data or attest to the source of the enclave rather than the precise software hash of the enclave.
In one embodiment once the signature on an authenticated enclave is verified, the public portion of the key used to sign the enclave is made available to the Sealing & Attestation mechanisms, allowing a vendor the ability to choose between the rigid protection based on the enclave measurement or more flexible protection based on the source of the enclave's code.
Enclave authentication is split into two parts. Each enclave is accompanied by an Enclave License with a signature chain rooted back to Intel. The enclave license indicates who the source/accountable entity for the enclave is, any special capabilities the enclave requires, and any additional information needed for identifying the particular business model/agreement that enabled this enclave. A license may be for a specific enclave, indicating the measurement of that enclave, or it may be for a key, which is then allowed to sign enclaves as needed.
For example, A could purchase a license authorizing them to produce enclaves for use in A's video player. To do this, Intel would create a license for the Vendor A's video player Root Key, along with capabilities that Intel permits Vendor A to use in video player enclaves. Vendor A will then use the video player Root Key to sign individual license files for each video player revision they release. This creates a license chain for the enclave may contain multiple intermediate licenses.
A chain of signed licenses is not ideal for evaluation during the enclave launching process, so instead they are combined into a single instruction digestible structure called a Permit. Permits are symmetrically authenticated using the CMAC algorithm and are interpreted during initialization (EINIT) of the enclave.
Most of the elements of the License are copied to the Permit, yielding similar structures. The License ID is a 64 bit number to identify a business agreement. License Type identifies what platforms this license applies to. A Bulk license allows this enclave to be launched on any platform supporting Secure Enclaves. A Per Platform license requires the platform to first contact the indicated License Authority, and request permission to launch the enclave. Once permission has been established, no further contact with the License Authority is needed, but this allows the License Authority to track the number of platforms this enclave is deployed at for billing purposes. The ISV that licensed this enclave may opt to establish a security version number for this version of the enclave. By doing so, data sealed by this version can be made available to future versions, but not previous versions. The flags field indicates flags for the enclave that may be set in order for this permit to apply. The Capability Mask is a bit mask of the special capabilities that this enclave may be granted. The ParentKeyHash is the hash of the public key that signed this enclave's license, hashed with the public key that signed that key. EntityHash is the expected hash of the entity this license applies to. In the case of an enclave, this is the value of MR.EADD for the properly constructed enclave. For a licensing key, this is the hash of the public key.
In a License, the public key used to sign the license is included in the license itself. The permit is MACed using CPU keys. A proper cpuMAC indicates that the EMKPERMIT instruction created this permit after validating the license chain back to Intel. If the LicenseType is not Bulk, then a licenseMAC indicates that the Architectural License Enclave has contacted the appropriate License Authority and has receive confirmation that this platform may launch the enclave.
Not all enclaves require a permit. In order to ease development for enclaves, permits will be optional during the development and debugging phases of the software's lifecycle. The following policies will be enforced by EINIT. Non-debug enclaves always require a permit to launch. Debug Enclaves will launch without a permit. However, if no permit is presented to EINIT, MR.Policy, ISV Sec Version, Permit Sec Version, and Capabilities will all be set to 0.
If a permit is used to launch a debug enclave, permit→Flags[DEBUG] may be set, and only capabilities allowed by debug enclaves may be set in the permit.
In order to launch an enclave, a permit may be created from the license that is shipped with the software, and then provided to the cpu to start the enclave. This process is broken down into three Permit Issuing, Additional License Approval, and Enclave Initialization.
A new instruction, EMKPERMIT, is used to create a permit from a license. EMKPERMIT creates a single permit from a single license, but can be called in succession in to convert a chain of licenses into a single permit with MAC using the Permit Key. The next section will describe this in further detail.
Each license includes a license type, which determines what additional steps may be taken for the permit to be usable. Per Platform Licenses require that a License Authority in the cloud maintain a billing count of platforms the enclave is deployed on. For licenses of this type, an additional step is required. An Architectural Enclave called the License Enclave will negotiate with the License Authority in the cloud, and upon approval, will provide an addition MAC on the permit using the License Key. Architectural Enclaves, for example, are always Bulk License, meaning they do not require the License Key MAC in order to run. They work on any platform supporting Secure Enclaves.
Permits are enforced at Enclave Initialization. During initialization the permit is processed, and if the enclave measurement matches that in the permit, and the MACs are correct, the enclave launches. EINIT will look at the license type and only inspect the License MAC for licenses requiring additional approval.
EMKPERMIT is a privileged instruction, due to the time required to verify the RSA signature on the license. This instruction takes a very simple signed credential, verifies it, and produces a permit from its contents. The license contains both a signature and the public portion of the key used to sign it. This allows uCode to only store a hash of the Intel's license signing key, and be able to validate Intel signed licenses. EMKPERMIT can also validate licenses signed by ISV keys, by providing an authenticated approval of their key. This is done by created a permit, which contains a hash of the ISV's public key. The result is that EMKPERMIT can verify Intel licenses using an internal hash, or ISV keys with a hash provided in a second permit.
EMKPERMIT takes 3 parameters: pointer to a License, an optional pointer to a key permit, and a pointer to an output permit. For Intel signed Licenses, the key permit is null, and an internally hardcoded set of permit parameters are used. The calling method is used to validate an Architectural Enclave's License and produce a permit for it. EMKPERMIT ensures that the public key in the license is authorize by the uCode (by comparing the hash of the included public key to the internal hash).
In the case of an ISV, an ISV's key will have a license signed by Intel. Calling EMKPERMIT without a key permit, will use the Intel key hash to verify the signature on the license and create a permit authorizing the ISV key's hash to represent a legitimate license signing key. EMKPERMIT is then called a second time including the ISV's key's permit. EMKPERMIT validates the key permit's MAC, and then uses the hash of the ISV key where it previously used the Intel hash. Assuming the public key in the enclave license hashes to the value in the ISV key, and that the enclave license is properly signed by it, EMKPERMIT will produce a permit for the enclave. This permit indicates the license information (which may be consistent through the entire chain), the hash of all the public keys in the license chain, the enclave's measurement, and its capabilities.
The following steps are taken by the u-code during EMKPERMIT:
The License Enclave is designed to make decisions about enclave launching outside the scope of visibility for uCode. For example, uCode cannot evaluate whether an ISV's business arrangements with Intel allow for an additional enclave deployment. The License Enclave is designed to collect whatever material is necessary to make an assessment and either further approve the enclave launch, or veto it. The License Enclave is only required to support complex business arrangements, and is not necessary for Bulk Licenses such as the ability to launch the enclave on any platform as many times as is needed.
The License Enclave is expected to be a system service. If a license indicates it needed further approval from the License Enclave, the chain of licenses and the enclave permit created by EMKPERMIT are passed to the License Enclave. The License Enclave then generates an approval request. The application then sends this approval request to the appropriate License Authority, which generates an approval notice. This is passed back into the License Enclave, and the License Enclave uses the License Key to MAC the permit in the licenseMAC field.
Once a permit is issued for an enclave, it may be evaluated and enforced by u-code in the enclave launch process. This is done as a part of the EINIT instruction, which takes the linear address of the permit as a parameter. The following additional steps are added to EINIT as part of the Authenticated Enclaves mechanism.
Capabilities
The current capabilities map is a 128 bit mask of capabilities available to this enclave.
The space is organized based on the action to be taken by EINIT. Bit 00-03 are reserved for future use as ring level restrictions are active on this enclave. 04-07 is reserved to indicate what page protections are permitted in the future. 08-23 are processor keys available through EGETKEY. 24-31 are for other controls, such as using Name Based mode for attestation or for future technologies we want to restrict. Certain capabilities may never be used by an enclave in debug mode. The Debug column indicates whether a capability is legal to use in Debug Mode.
In future generations, bit 00 may indicate that ring level and VT restrictions apply to this enclave. Bits 01-02 indicate what ring level the enclave is permitted to run at, and bit 02 indicates whether the enclave runs in VT root mode or not. On each EENTER the current CPL may be compared against bits 01-02 to determine if this enclave is allowed to execute at this ring level. If an attempt is made to execute it at the wrong ring, EENTER will fail. Similarly, if ring restrictions are active, the enclave may only be entered from VT root mode if bit 03 is on. In the first generations these bits are MBZ.
Enclave pages may be encrypted or only integrity protected. Also, pages may be executable or not. In future generations, these attributes may be tracked and enforces in the security info portion of the EPCM. These capability bits are reserved to control the application of encryption to enclave pages in the enclave based on whether the page is executable and whether the enclave has been EINITed already.
Many Architectural Enclaves are Ring 3 entities that require access to keys protected within or by the CPU. EGETKEY provides access to these keys while the capability bits are used by EGETKEY to decide if access to the key may be granted.
The following is a list of the current Architectural Enclaves with their properties and short descriptions.
The Provisioning Enclave, with capabilities KEY_PROVISION and authorized by Intel, runs on single package platforms whenever a new Device Attestation Key (DAK) or Provisioning Attestation Key (PAK) is required. Its purpose is to allow the enclave to derive Device ID & Provisioning Key based on the Provisioning Seed provided by EGETKEY. The Provisioning Enclave then uses these keys to prove the authenticity of the platform to a provisioning server and retrieves a Device Attestation Key (DAK). After retrieving the DAK, the Provisioning Enclave seals it such that the Quoting Enclave can retrieve it. The Provisioning Enclave may then optionally use the DAK to authenticate with a Platform Attestation Key (PAK) provider and retried a PAK. Using a PAK provides better privacy for the user by ensuring that for a particular ISV, their activities cannot be associated with those of a previous owner of their platform. After retrieving the PAK, the Provisioning Enclave seals it such that the Quoting Enclave can retrieve it.
The Quote Enclave, with capabilities KEY_REPORT and authorized by the enclave has the same author as the Provisioning Enclave (typically Intel) used to provision the EPID key. Its location is OS Service Available to all apps. Its purpose is to allow enclaves to unseal a platform EPID key. A Report from EREPORT is provided as input. The enclave uses EGETKEY to retrieve the Report Key. The Report key is then used to verify the report. Enclave signs a Quote from using EPID.
The License Enclave, with capabilities KEY_LICENSE and authorized by Intel and signed by Root Intel, is shipped with Enclaves (OS Service) and singularly instantiated. Its purpose is to evaluate complex license policies. If an enclave requires additional license confirmation from the License Enclave, EINIT will only accept it after the License Enclave uses the License Key to CMAC the permit.
In single-package systems all the symmetric keys used by the enclave's architecture are derived from a single source of uniqueness stored in the processor's fuse array. The key hierarchy is split into an SE TCB Hierarchy, which is platform implementation dependant, and a SE Key Hierarchy whose structure is consistent across all Secure Enclave implementations. Keying material for TCB recovery and the foundation of EPID provisioning is provided by the SE TCB Hierarchy which serves as the root for the SE Key Hierarchy. All keying material used both within the enclave instruction set and in trusted Architectural Enclaves is provided by the SE Key Hierarchy.
The platform provides a two 128 bit platform unique keys in fuses. These keys are encrypted in fuses using a key stored in secret CPU logic. Several single purpose keys are derived from this key, and TCB recovery techniques are applied based on the platform's requirements. The resulting keys serve as the roots in the SE Key Hierarchy.
Keys for the Architectural Enclaves are retrieved using the EGETKEY instruction.
The enclave architecture also requires the use of an asymmetric key to provide attestation of the REPORT values to systems outside the platform. This key, an EPID key, is initially provisioned in fuses, but may be re-provisioned using one of the keys derived from the key hierarchy after deployment. The method for provisioning the EPID attestation key is outside the scope of this specification. More information can be found in the Device Attestation Key (DAK) Provisioning Specification.
Finally the enclave's architecture also makes use of a key which is in the logic of all processors, for provisioning of key material at the OEM. This key is known as the Out-of-Box Experience Global Key. We perform similar derivation operations on this key to provide ISV uniqueness. How these keys derived from the OOB Key are used by ISV's is beyond the scope of this specification.
While the SE TCB portion of the key hierarchy is platform specific, all foundations require the same basic set of keys. We refer to these as the base keys. They are all derived in a fuse key and a logic key, and are the root of the SE Key Hierarchy. These keys are then used by an SE instruction to derive all keys used directly in the SE architecture. These keys are the result of the TCB Key Hierarchy. There are four SE Base Keys plus EPID components which are made available to the SE architecture by platform specific mechanisms. Table 12-1 describes each of these keys.
The Secure Enclaves instructions and data structures rely on the Base Keys as a source for keying material. The Platform Key Hierarchy shown in Table 12-1 describes the hierarchical relationship of the platform key material and how keys are derived from the Platform Root Key.
The Enclave Wrapping Key, 1752 is a symmetric key used to encrypt the Secure Enclaves Control Structure (SECS) page while it is not protected inside the Enclave Page Cache (EPC). This key is only used by uCode.
The Permit Key, 1754, is used to provide authenticity and integrity over Permits, which contain capability and licensing information for an enclave. Permits are MACed to ensure their integrity while in transit to EINIT. This key used by EMKPERMIT uCode and EINIT.
The License Key, 1756, is used assert compliance with license policies not able to be evaluated by uCode. The License Key is used to produce an authenticated approval from the License Enclave that is evaluated by EINIT. This key used by EINIT uCode, and is available via EGETKEY to enclaves with the KEY_LICENSE Capability set.
The Report Key, 1758, is used to provide authenticity and integrity over Reports. Reports are MACed by the ERPEPORT to ensure their integrity while in transit to the Quoting Enclave. This key used by EREPORT uCode, and is available via EGETKEY to enclaves with the QUOTE Capability set.
The Auth Key, 1760, is an enclave specific key, and is used to provide authenticity and integrity over data transmitted from the Quoting Enclave to an ISV Enclave and enables enclave-to-enclave authentication on the same platform. The key is available via EGETKEY to all enclaves, and those enclaves with the ISV_AUTH Capability set can specify which key it requires.
The Seal Key, 1762, provides each enclave with a 128-bit key to encrypt their sensitive data. A number of sealing policies can be integrated into the seal key, providing ISVs with flexibility on what software can unseal their data. These keys are available to any enclave via EGETKEY, but individually a seal key is only available to an enclave that meets the seal policy requested.
The EPID ID, 1712, uniquely identifies the package. Its sole purpose is to enable the provisioning of Device Attestation Keys, which are EPID-based anonymous attestation keys. The EPID ID is only accessible to the provisioning enclave. The provisioning enclave will only provide it over a secured channel to an approved provisioning server, and only during the provisioning process, which is initiated by the user or operating system. This ID is available via EGETKEY to enclaves with the PROVISIONING capability.
The Provisioning Key, 1710, is used to prove authenticity of a platform to the Intel Backend, as well as to authenticate the current SE TCB running. By demonstrating access to the Provisioning Key, the provisioning server is assured that the enclave is indeed the device in possession of EPID ID, and is running at least the specified TCB security version. The Provisioning Key is unique to this package and the signer of the provision enclave requesting it. This creates a separation between provisioning infrastructures, if more than one is used on a single platform. This key is available via EGETKEY to enclaves with the KEY_PROVISION capability.
The Provisioning Seal Key provides the provisioning enclave with a 128-bit key to encrypt provisioning in a way that can be retrieved even after a change of ownership. This key is used to encrypt old EPID in order to prove the platform has not been revoked while acquiring new EPIDs. The Provisioning Key is unique to this package and the signer of the provision enclave requesting it. This creates a separation between provisioning infrastructures, if more than one is used on a single platform. This key is available via EGETKEY to enclaves with the KEY_PROVISION capability.
The ISV Out of Box (OOB) Experience Key, 1700, is a shared key between all Intel platforms and an ISV. This key is derived from the OOB Root uniquely to a specific ISV. ISVs will be able to purchase access to this key, allowing them to encrypt secrets to this key and placed in an OEM's hard disk image. These secrets will only be accessible to their code running safely in a secure enclave, and does not require the platform to go online or complete attestation key provisioning. These keys are available via EGETKEY to enclaves with the OOB Capability.
Provisioned keys are those critical keys to the Secure Enclave architecture, but are not derived from the platform keying material. These keys are provisioned from a provisioning server or offline techniques. The Device Attestation Key (DAK) is an anonymous signing key (EPID) use to attestation to the properties of individual enclaves. This can be used by an ISV during key or secret provisioning to ensure that sensitive information is only sent to protected instantiations of their untampered applications.
There are two sources for the Device Attestation Key. The preferred architecture ships with an initial DAK compressed in fuses as the EPID Key Blob and EPID Entropy. This enables the platform to perform attestations immediately after the first power on. The second source is by contacting the DAK provisioning server and downloading one after proving the legitimacy of the hardware using the EPID ID and Provisioning Key. This second method is used by platforms, which do not have fused EPID keys as well as any platform after we revoke a version of the underlying TCB. The EPID fuses are accessible via EGETKEY to enclaves with the PROVISIONING capability.
The Platform Attestation Key (PAK) provides an optional additional level of privacy. Certain uses of the DAK can be associated. Specifically if an ISV enclave has the Name Based Attestation capability, then that single ISV can determine if a given EPID is revisiting that service. (Multiple ISVs cannot collude to track users, however). Since the DAK is bound to the platform, rather than the owner, this association continues through waterfall events. Therefore some users will prefer to use their DAK to assert the legitimacy of their platform to a third party that will issue a PAK to use for daily attestations. In multi-package platforms the DAK's of each package is used to establish the PAK, which represents the whole of the platform in attestations.
Key derivation for user accessible keys shall comply with NIST Special Publication 800-108 (Recommendation for Key Derivation Using Pseudorandom Functions). In the construction of a key derivation function, a Pseudorandom Function (PRF) is needed. The PRF shall be based on the AES-CMAC algorithm as defined in NIST SP 800-38B, Recommendation for Block Cipher Modes of Operation—The CMAC Mode for Authentication, May 2005. (http://csrc.nist.gov/publications/nistpubs/800-108/sp800-108.pdf). The key derivation generally looks like the following:
Derivative Key=PRFParentKey(Derivitive String)
The derivative string is composed of a subset of 8 elements based on the specific key being requested. Table 12-2 describes each available element that may be part of a derivation.
Each key has a predefined set of derivation elements, which will compose the derivation string. Table 12-3 describes which elements are included in each of the keys from the key hierarchy. Each column represents a key, and the rows indicate whether a specific element is included in that key. The Debug string is included if the SECS of the requesting enclave indicates it's in debug mode, and “Request” indicates that this element is not required, but is selectable in the request to derive the key.
Secure Enclaves supports techniques for isolation and recovery of software compromise at several points in the boot sequence. In order to support isolation, all long term keying material provided to enclaves is derived using the security versions of the current TCB.
This section describes an example architecture for a platform whose recoverable TCB is composed of uCode, MCHECK, and microcode (or a uVMM) will be described. The hardware requirements are the same for any SE supporting platform, however the exact key flow is dependent on the specific TCB elements. Other platforms can be supported using similar techniques to those applied here.
The following keys are required in hardware to support a CPU-based protection technology. These keys are the foundation of the TCB Key Hierarchy.
Stepping-Specific 256-Bit Logic Key:
The 256-bit logic key is broken into two parts—128-bit fuse wrapping key, and 128-bit out-of-box experience key. It is possible to use a single 128-bit key for both, however, that adds more uCode.
Die-Specific 544 Bits of Fuse Key:
These include 32 bits of group id, 256 bits of SafeId A.x value, and 256 bits of pre-seed. The A.x value and the 256-bit pre-seed are encrypted with the 128-bit fuse wrapping key described above.
Temporary Registers:
The key-derivation process requires the keys be stored and on the package and available only to uCode. Two 128 bit registers are needed for the duration of platform runtime. An additional 256 bits of space are needed for the EPID key until CMA is up and running After which the additional 256 bits are no longer needed in the CPU.
TCB SVN Register:
This register is a 64 bit lockable register that is sub-divided to hold SVNs for each TCB layer. Specific subdivision is at the discretion of the platform designers, but 8 8 bit SVNs would be reasonable. Each section of this register may be independently lockable.
The binding of keys to a specific set of TCB version is achieved by having the uCode derive a first set of keys from the fused key, based on the type of boot sequence that will commence. After this the fuses are locked, and a chain of derivations occurs at each load in the boot sequence.
After the low level code is loaded, the chain continues to include the ISV assigned security version for the software running in the enclave. For any specific configuration, keys derived from the current version are accessible, as well as keys from previous configurations. This enables seamless user data transitions to newer non-vulnerable versions.
Once the die-specific key is generated, it is encrypted with the key wrapping key. This increases the difficultly of extracting the keys with hardware monitoring tools as well as provide protection for the keys in transit before being deposited in the part.
The crypto algorithm used to encrypt these keys is 10 rounds of 128 bit AES-ECB decrypt. The key generation server will apply AES-ECB encrypt to each key to generate a cipher text key that will be burned in fuses.
The Pseudorandom Function (PRF) used for key derivation in the TCB Key Hierarchy is platform specific. We recommend 128 bit AES-ECB for platforms supporting AES-NI. The goal is to provide a non-reversible way to derive keys from other keys. For this section we use the following functional prototype.
void PRF(uint128*key,uint128*string,uint128*key_out){
There are three ways to PRFs are used in key derivation. The PRF Loop Derivation is used to inject the uCode SVN into a key, while establishing a relationship between keys of different SVNs. Specifically:
PRFLoop(x−1)=PRFPRFLoop(x)(const)
This provides forward migration of data. Take the example of running uCode SVN 3. An enclave uses EGETKEY to retrieve a seal key based on this version (PRFLoop(3)), and seals data with it. An in the field uCode upgrade is delivered and the next boot the uCode SVN is 4. After the upgrade, the EGETKEY implementation will have access to PRFLoop(4). When the enclave requests the SVN 3 key from EGETKEY, it can compute PRFLoop(3)=PRFPRFLoop(4)(constant) and therefore retrieve the old seal key.
In order to establish this property, a loop of PRFs is used, however because of the property PRFLoop(x−1) is computed from PRFLoop(x), we need to establish a maximum SVN and count back from it. The specific maxes will need to be established for each platform type based on required performance. We recommend a 32 as the initial max point.
Application of a PRF Loop Derivation generally looks like the following:
This method will be used to inject uCode's SVN into the SVN key, which will be the underlying key behind the SE base keys. The die-specific key in fuses contains 288 bits of EPID values and a 256 bit of random key. All non-ephemeral symmetric keys may be derived from these 256 bits, which is composed of 2 128 bit keys. Therefore a technique may be created for deriving multiple keys from a single key. To do this, after the fuse key is decrypted, we use it to call PRF using different fixed constants.
Application of a key splitting generally looks like the following:
This technique is used to generate random numbers used as part of the EPID ID and a provisioning ID.
Once the SVN key has been loop derived based on the uCode SVN, it can be store away in protected memory such as the SE CMA. Microcode will use an MSR exposed to microcode only to derive keys from the SVN Key. The MSR takes a key selector that indicates whether the basis for the derivation is the global out of the box key or the fuse key, and a set of requested SVNs for each TCB layer. It verifies the request is less than or equal to the current values. UCode applies any necessary PRF's to retrieve an old SVN keys, and the PRFs the requested TCB SVNs.
svn_base_key=PRF(tmp_svn_key,requested_tcb_svns);
Once the proper SVN key is available, it is used as the key for a CMAC over the requested TCB SVN's. Microcode then uses this as a CMAC key over the SE Ops Seed (a value derived from the portion of the fuse key not known by Intel) for the Ops key, or a fixed string for the Provisioning Base Key.
se_base_key=CMAC(svn_base_key,se_ops_seed);
All cores synchronize and ensure they are all in MCHECK using doorbells or similar mechanisms. Once all the cores are executing MCHECK, following steps are taken by the BSP. AP's do not participate in the key flow:
Either during microcode initialization or upon calling EGETKEY, microcode calculates the SE Base Keys needed to satisfy requests. The Base Keys may be cached in the CMA for further use for increased performance. Table 12-4 describes how the base keys are computed.
In order to protect user privacy and data across platform waterfalls, a 256-bit random Owner Epoch is included in the derivations of key. This value is created randomly during ownership change. Prior to the use of enclave keys software may write the OwnerEpoch to the SE_EPOCH_MSR. This can be achieved by the BIOS, which stores it persistently in flash. It can be calculated from some user input, such as the hash of a user boot password. It can also be provided by a Secure Enclaves driver prior to allowing enclave use.
Confidentiality of this value is required to ensure that data encrypted by the platform cannot be decrypted in the originally authorized enclave by someone in possession of the laptop after a waterfall. Compromise of this value does not result in the compromise of any enclave data.
The SE Key Info structure is a non-persistent structure stored in a protected area of memory or the package. The CMA is the most likely location, but any on die protected storage is also ok. During Power on, the SE Key Info is initialized. KeyID is set to a random value, and Key Count is set to 0. On each use of the Enclave Key, Permit Key, and Report key the KeyID read, and the Key Count is incremented. After 2^32 key uses, the KeyID is changed to a new random value, and Key Count is reset to 0. The SE Key Info layout is shown in 5.
On power-on, the Platform Key Table is initialized by uCode. BIOS or other host firmware acquires the current Owner Epoch either from persistent storage or from the user and writes it to the LoadOwnerEpochMSR. At this point the enclave key hierarchy is available.
Much of the enclave's architecture relies on the use of keys to provide authentication and confidentiality of enclave data, and in order to keep the processor complexity to a minimum Architectural Enclaves are used to process these keys for high level usages. E.g. The Quoting Enclave uses the REPORT key to establish that a REPORT structure generated by the EREPORT instruction was created on the platform, and the PERMITING enclave uses the PERMIT key to create an enclave PERMIT which is consumed by EINIT when an enclave is being launched.
In addition any application level enclave needs access to a key to seal secrets which are stored on the platform outside the enclave, and will be unsealed when the application enclave is re-established—even across power cycles.
The mechanism for doing this is the EGETKEY instruction. It is the single interface for establishing secrets about the current software environment.
EGETKEY currently provides access to the following keys:
Most of these values do not reside in the processor in the raw, but are in fact derived on demand by EGETKEY from a single fuse key value. They are derived on demand as each of these keys are not a single key but are in single key from a possible set. The particular key delivered depends on a number of parameters, some of which are user selectable, others are based on the system or particular state.
In order to select the key a KeyRequest structure is used as an input to the EGETKEY instruction. As well as selecting the key the user wants the KeyRequest structure allows the caller to specify those variables under his control which he wishes to be used in the creation of the key. The figure below specifies the KeyRequest structure:
KeySelect is used to identify the key the user requires, and KeyPolicy is used to establish which additional values are used in creating the key—whether that be a particular security version of the architectural enclaves, or a particular version of an application enclave, or the measurement registers associated with the current enclave (when EGETKEY is called from within an ENCLAVE).
Additional randomness can also be added to the key derivation, this particularly required to prevent wearing of keys, and is used by the PERMITING and QUOTING architectural enclaves. It would also be used by the application enclave when creating SEALing keys. Setting the field to zero indicates that no additional randomness is to be added, otherwise the field points to a 256 bit aligned data value. The figure below specifies the structure for the KeySelect field.
KeyPolicy is a bit field selector and is used to determine if a particular value, either from the user or the system state is to be used in deriving the key.
Enclave Registers and Control
Two enabling levels are provided for enclaves. The first enable is an opt in bit set by the BIOS. It is a write once function. It enables or disables enclave capability until the next reset. The second enable is provided to the OS or VMM to turn enclave capabilities on or off dynamically as needed.
The Enclave capability is enabled by first setting the Enable bit in the EnclaveCTL_MSR shown in
Software can detect support for enclaves by executing the CPUID instruction. CPUID will return a result indicating whether enclaves are supported or not.
If the Opt in bit is cleared then CPUID reports that enclaves will not execute.
System software controls enclave capability using the EnclaveCTL_MSR shown in
Software can detect support for enclaves by executing the CPUID instruction. Enclave support is indicated if the ON bit in the EnclaveCTL MSR is set
The TCSMSR register is a register on each processor which contains the address of the TCS. It is used by exception handling and the RDTCSPTR. It is loaded when entering the enclave. The register is loaded with the value of the TCS when EENTER is executed. It is read by ERDTCSPTR. The register size is based on the mode of the processor.
The enclave base address register on each processor contains the lower address of the enclave under execution. It is loaded when entering the enclave by the microcode. The register size is based on the mode of the processor. This register is not visible to the software. It is a microcode temporary.
The register holds the upper address limit of the current enclave. It is loaded when entering the enclave. The register is loaded with a value stored in the SECS when the enclave starts execution. It is a microcode temporary register. Register size is based on the mode of the processor.
The Enclave Page Cache (EPC) Maximum Size Register indicates the maximum size of the EPC. This size is given in the number of 4096 byte pages. It is a 32-bit register. This register is read only to indicate the largest size EPC supported in the current design.
The EPC Size Register EPC_SIZE MSR indicates the currently defined size of the EPC. Loading the register results in an EPC defined to the size. The value is given in 4096 bit pages. For example, one 4096 bit page would be a 1. The value of the register cannot exceed EPC_MAX value. If the value exceeds the EPC_MAX value a GP fault is taken by the WRMSR instruction. Writing to this register will invalidate all data in the EPC prior to the write. Software may save all EPC entries (if needed) before updating this register.
The EPC base register indicates the location of the base of the EPC. Writing to this register will invalidate all data in the EPC prior to the write. Software may save all EPC entries (if needed) before updating this register.
In general no external interfaces shall allow any transfer or transaction which can compromise the security of enclaves. Secure Enclaves requires random numbers for the enclave keys. Random bits are generated using the digital random number generator. The random number generator may be securely accessible by the microcode. It does not need be located in the part's core.
An enclave can be set as a debug enclave when it is created. The debug enclave will allow external access to the enclave contents using the EDBGRD and EDBGWR instructions. A debug enclave is set up by setting the debug flag in the ECREATE instruction. This bit is stored inside the SECS of the enclave.
Enclaves which are created with the debug bit clear are production enclaves. The EPC contains a debug bit which indicates that the enclave is a debug enclave. The enclave remains encrypted inside main memory or on disk. Debuggers needing to look at the enclave contents will load the memory into the EPC. The EDBGRD and EDBGWR instructions can be used to access enclave memory locations which reside in the EPC. A debug enclave does not require a permit in order to execute. It will execute without a valid permit.
When entering a production enclave the debug control register, DR7 is saved in the TCS save area. DR7 is shown in
Bits L3-L0 and G3-G0 are set to zero value. DR7 is returned to its original value at enclave exit.
For debug enclaves, the debug register value is not changed. When RFLAGS.TF is set at the start of an EENTER instruction, there are two cases to be considered:
In the first case, the #DB exception may occur on the target of the next EEXIT instruction. This treats the enclave as large, opaque operation. In the second case, the user has complete freedom to single step through the enclave. This behavior is supported by 3 data fields inside the enclave and special processing of RFLAGS.TF on EENTER, EEXIT and EIRET.
The register value is saved in the TCS save area. The register is set to 0. At enclave exit the register is restored to the value at entry. If the enclave has branch trace enabled at entry the EENTER is the last entry before entering the enclave. When exiting the enclave the first location after the exit is written to the branch trace.
Int n and Int 3 instructions are reported as GP faults if executed inside the enclave. The debugger may hook the GP fault condition when debugging an enclave.
In this document we describe a novel technique for implementing the CMAC mode of operation for the AES block cipher. CMAC is a mode supporting message authenticity. It accepts as input a message A and a key K and returns an authentication tag T. The derivation of the authentication tag is done using the CBC (cipher block chaining) algorithm. CMAC is more complex than CBC because it includes mechanisms for protecting against length extension attacks. We refer to these as the ‘three peculiarities of CMAC’. In what follows we provide an overview of CBC and CMAC.
The CBC algorithm utilizes a block cipher to provide confidentiality for some piece of data or to compute an authentication tag on this data. The main idea behind the CBC algorithm is that the output from the previous encryption is XOR-ed with the next input block before being encrypted. In this way patterns which may exist in the input data are eliminated in the ciphertext. Also the combination of the XOR operations between and the transformations of the block cipher provide strong mixing for deriving a message authentication tag which ideally is not forgeable.
The CBC algorithm is given below and illustrated in
The CMAC specification includes three additional algorithms for initializing and finalizing the CBC algorithm. We refer to these as the “three peculiarities” of CMAC. The first peculiarity concerns the derivation of two subkey values K1 and K2 from the symmetric key K. Subkeys K1 and K2 derive from an intermediate value L. CMAC specifies that L derives by applying the symmetric key block cipher transformation on a string consisting of zeros (i.e., 0128) using the symmetric key value K. Such relationship is shown in equation (1):
L=CIPHERK(0128) (1)
Once L is derived the most significant bit of L is checked. If this is zero then K1 derives from L by shifting by one bit position to the left. Otherwise L is shifted by one bit position to the left and also XOR-ed with a special value Rb to produce K1. Rb is defined as <012010000111> in binary form. K2 is produced from K1 following the same procedure. The derivation of subkeys K1, K2 is given in pseudocode below. By MSB( ) we mean the most significant bit of a value.
The second peculiarity of CMAC concerns the padding that takes place before applying the CBC algorithm on the input data. If the last block of data is not a complete block then the block is padded with a bit equal to “1” followed by as many zero as needed so that the final block becomes complete.
The third peculiarity of CMAC concerns a modification on the last block that takes place in order to avoid length extension attacks. If the last block is a complete one (no padding required) then the last block is XOR-ed with the subkey K1. Otherwise it is XOR-ed with the subkey K2.
The algorithms for CMAC tag generation and validation are listed below:
In what follows we show how one can implement the CBC 0 algorithm when the symmetric key block cipher used is AES and the processor supports a set of instructions for AES round acceleration. The Intel Architecture supports 4 new such instructions at the time frame of the Westmere processor (2009) and on. These instructions are AESENC (AES round encryption), AESENCLAST (AES last round encryption), AESDEC (AES round decryption) and AESDECLAST (AES last round decryption). The specification for these instructions is as follows:
To implement the CMAC mode using the AES round instructions it is sufficient to invoke AESENC AESENCLAST only since the tag validation process is the same as tag generation.
To implement the key schedule transformation one can use the AESIMC instruction for inverse mix columns and AESKEYGENASSIST instruction AESKEYGENASSIST is used for generating the round keys, used for encryption. AESIMC is used for converting the encryption round keys to a form usable for decryption according to the equivalent inverse cipher model. The description of the AESIMC and AESKEYGENASSIST instructions is given in http://softwarecommunity.intel.com/articles/eng/3788.htm.
CMAC is specified using the big endian notation for the 128-bit quantities involved. To implement CMAC in a little endian machine correctly one needs to perform 16 byte-wide byte reflection operations at certain points in the source code implementation. Such operations can be quickly performed using the PSHUFB instruction (1 clock latency, throughput). In what follows we describe the points where byte shuffling is required.
In the SUBKEYS( ) algorithm implementation byte reflection is required on L after it is derived by applying the AES cipher on the zero string and before the derivation of the two subkeys. Also byte reflection is required on the two subkeys after they are derived from L. A SUBKEYS( ) implementation is given in C below:
Next, byte reflection is required on the last block before and after padding only if this last block is not complete. These steps are shown in a C implementation below:
Where the function _pshufb( ) performs 128-bit wide byte reflection.
SE Requirements for SMI
Enclaves are not allowed to execute inside SMM space. An attempt to execute an enclave in SMM mode will result in a GP fault of the instruction. When an SMI occurs while executing inside an enclave, the processor may save the register state away inside the enclave and exit. When the exit occurs the TBD MSR bit is set to indicate that the SMI occurred while the enclave was executing. The SMM code cannot access the enclave data. Attempts to touch the EPC area will result in the return of junk data in real mode and an EPC page fault in protected mode.
Certain instructions are not allowed to execute. There are a number of general rules used to decide which instructions are legal.
In the first generation of enclaves the processor may be running in ring 3 with the IOPL set to 0 when entering an enclave. In order to preserve the programming environment when enclaves run either a virtualized or non virtualized environment, the instructions listed in Table 18-1 are illegal.
Restrictions are imposed on the state inside an enclave. When entering the enclave, the GDTR.limit, LDTR.limit, IA32_EFER.SCE, and IA32_SYSENTER_CS are saved in the TCS area. The local values are cleared. Instructions which access or cause access to these register will fail inside an enclave. The GDTR.limit, LDTR.limit, IA32_EFER.SCE, and IA32_SYSENTER_CS are restored when leaving the enclave.
The life of an enclave is divided into distinct phases. The first phase is enclave creation. The second phase is enclave usage. The final phase is enclave destruction.
The creation and usage of an enclave requires the support of the OS/VMM. While the enclave will not depend on the OS/VMM for security, it will require the OS/VMM to properly maintain certain hardware data structures. Failure of the OS/VMM to maintain these structures will not result in a loss of security, but may cause the total failure of the enclave.
Several instructions support attestation of enclaves, sealing and unsealing of secret data and the permitting of authenticated enclaves.
In the first phase, the enclave may be securely constructed and the internal software environment set up for use by the application. Three instructions are used to create the enclave. The first instruction, ECREATE, sets up the initial state environment. This instruction creates the enclave key, loads, encrypts, and integrity checks two pages used to store the enclave data structures. The second instruction, EADDPRE, adds a page of data to the enclave. It adds pages needed for the code, stack, and heap inside the enclave. The third instruction, EINIT, sets the internal software environment to a known state. At the completion of this instruction the enclave has moved to the second phase, usage.
Before performing EINIT the construction software may have obtained a permit, either by performing EMKPERMIT or by using the permitting enclave.
The enclave is entered through the EENTER instruction. This instruction transitions the machine into enclave mode. It transfers control to a predefined entry point. The EEXIT instruction returns from the enclave to the outside application. The EIRRET instruction returns into the enclave from an interrupt exit.
When entering the enclave via either EENTER or EIRET the following operation is performed by the instructions. Save and Clear GDTR.limit, LDTR.limit, IA32_EFER.SCE, and IA32_SYSENTER_CS. On exit restore GDTR, LDTR, IA32_EFER, and IA32_SYSENTER_CS.
There are no instructions for destroying an enclave.
EDBG_READ instruction does an 8 byte read of a location inside a debug enclave. No access is allowed to non debug enclave. EDBG_WRITE instruction does an 8 byte write to a location inside a debug enclave. No access is allowed to non debug enclave.
The Enclave Page Cache (EPC) is managed via 2 instructions. Two instructions load/store EPC pages (ELPG and EWBINVPG).
EREPORT generates a cryptographically protected structure that holds an enclave measurement. EGETKEY provides a means of retrieving an enclave specific key of vary types. EMKPERMIT is used to create a permit for an authenticated enclave.
1No usage model for INSIDE, but no known harm in allowing EMKPERMIT to execute from inside.
2Future version may enable entry into enclaves from ring 0.
On interrupt, the processor state may be saved (and hidden) inside the enclave and the state then cleared. Furthermore, even the return address of the interrupt may be hidden.
Interrupts occurring while an enclave is executing may push information onto the interrupt stack in a form expected by the operating system so as to avoid the need to change OS code. To accomplish this, a pointer to trampoline code is pushed onto the interrupt stack as the RIP. This trampoline code eventually returns to the enclave by means of an EENTER instruction with a special parameter (q.v.).
The interrupt stack to be used is chosen using the same rules as for non-SE mode:
In all cases, the choice of interrupt stack and the information pushed onto it is consistent with non-SE operation.
The TCS::IRR (Interrupt Return Routine) points to a per-thread code sequence that will later return to a specific thread. This pointer is pushed onto the interrupt stack as the return RIP. This results in a set of data structures that causes the IRET to return to the application where the interrupt return code (which includes the specialized EENTER instruction) is executed. The EENTER takes the RBX register initialized at the time of the interrupt (and preserved by the OS) and uses it as a TCS to re-enter the enclave.
The following bits in RFLAGS are cleared before the register is pushed onto the interrupt stack:
The State Save Areaholds the enclave state at the time of an interrupt. Because an interrupt may be delivered to user mode that may then re-enter the enclave, the SSA is a stack of multiple SSA slots as illustrated in
When an interrupt occurs while executing on a thread inside the enclave, microcode determines the Save Area to use by examining TCS::SSA and TCS::CSSA. Processor state is saved and cleared (to avoid leaking secrets) and TCS::CSSA is incremented. As will be described later, if an exception takes the last slot, it will not be possible to deliver the exception to the enclave.
Note: On EENTER, CSSA may be <NSSA, ensuring that there is at least one Save Area available for interrupts (unless EENTER is being used to return from an interrupt).
On any transition to the Interrupted state, untrusted code (either the application, the OS or both) may only resume the enclave using EENTER/RETURN_FROM_INTERRUPT.
On any transition to the Excepted state, untrusted code (either the application, the OS or both) may decide to:
EENTER in Excepted state advances to the Handling state. EEXIT from the trap handler (Handling state) advances to the Handled state. ENTER/NORMAL is illegal in this state. EIRET from the trampoline resumes the state pushed onto the SSA at the time of the last interrupt; either the Active or Handling state.
Secure Enclave Instructions are split into 2 opcodes, a privileged opcode and an unprivileged opcode. The instruction operation is determined by the value in RAX at the time the instruction is called.
The ECREATE instruction initializes a protected SECS. The source operand points to a page_info structure. The content page field points to an unprotected SECS structure. The SECS structure may be page aligned. The lower 12 bits of SECS base and bound values may be 0. The SECS is an address of an empty slot in the EPC. The sec_info is an address of an unprotected sec_info structure. The corresponding sec_info flag field may be properly initialized.
Instruction Input(s)
Instruction Output
Register RAX.RAX contains the error code if instruction fails.
Instruction Operation
Flags Affected
None
Use of Prefixes
Lock: causes UD#
REP: causes US#
Segment overrides: N/A
Operand Size: causes UD#
Address Size: ignored
Rex: ignored
EADDPRE
Instruction Description
EADDPRE allows privileged software to copy a page outside of an enclave to a page within an enclave specified by lin_addr and the attributes of the enclave page are set using the sec_info flag field.
As part of the instruction the page is hashed and the resulting hash value is extended into the enclave's measurement register.
EADDPRE can only be performed on enclaves that have not yet been initialized by the EINIT instruction.
Instruction Inputs
Instruction Operation
Flags Affected
None
Use of Prefixes
Lock: causes UD#
REP: causes US#
Segment overrides: N/A
Operand Size: causes UD#
Address Size: ignored
EADDPOST
Instruction Description
EALLOCATE allows privileged software to initialize an SMAP entry of an enclave specified by lin_addr and the attributes of the enclave page are set using the sec_info flag field.
Before the enclave can access the page, it may accept the page into the enclave using the EACCEPT instruction.
EALLOCATE can only be performed on enclaves that have been initialized by the EINIT instruction.
Instruction Inputs
Instruction Operation
Flags Affected
None
Use of Prefixes
Lock: causes UD#
REP: causes US#
Segment overrides: N/A
Operand Size: causes UD#
Address Size: ignored
Protected Mode Exceptions
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EMKPERMIT
Instruction Description
Authenticates an enclave or license and generates a permit from it. If rbx==NULL, certificate may be signed by Intel. Otherwise the license may be signed by the key indicated in the rbx permit.
Instruction Inputs
Instruction Operation
Flags Affected
None
Use of Prefixes
Lock: causes UD#
REP: causes UD#
Segment overrides: N/A
Operand Size: causes UD#
Address Size: ignored
Protected Mode Exceptions
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EINIT
Instruction Description
EINIT marks the enclave as ready to run in a software environment. At the end successful initialization, EENTER will be allowed for the enclave.
Instruction Inputs
Instruction Operation
Flags Affected
None
Use of Prefixes
Lock: causes UD#
REP: causes UD#
Segment overrides: N/A
Operand Size: causes UD#
Address Size: ignored
Protected Mode Exceptions
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
ELPG
Instruction Description
This instruction is used to load a page into the Enclave Page Cache (EPC).
Instruction Inputs
Linear address of the source page: page_info
Linear address of the destination: epc_la
Instruction Operation
Return:
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EWBINVPG
Instruction Description
This instruction is used to write-back dirty pages from EPC to the main memory.
Instruction Inputs
page_info
epc_la
Instruction Operation
Return:
If successful, the instruction sets the Z flag in the ELFAGS register to 0, otherwise the Z flag is set to one. The ENCLAVE_STATUS_MSR holds one of the following reasons:
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EUPSMAP
Instruction Description
This instruction checks and updates the version of enclave page resident in the EPC.
Instruction Inputs
leaf
epc_la
smap_la
Instruction Operation
Return:
If successful, the instruction sets the Z flag in the ELFAGS register to 0, otherwise the Z flag is set to one. The ENCLAVE_STATUS_MSR holds one of the following reasons:
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EREMOVE
Instruction Description
This instruction updates the SMAP when data is loaded into the EPC.
Instruction Inputs
leaf
epc_la
smap_la
Instruction Operation
Return:
If successful, the instruction sets the Z flag in the ELFAGS register to 0, otherwise the Z flag is set to one. The ENCLAVE_STATUS_MSR holds one of the following reasons:
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EADDSMAP
Instruction Description
This instruction is used to add a new page to the SMAP when the enclave is already initialized.
Instruction Inputs
Instruction Operation
Return:
If successful, the instruction sets the Z flag in the ELFAGS register to 0, otherwise the Z flag is set to one. The ENCLAVE_STATUS_MSR holds one of the following reasons:
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EMODIFY
Instruction Description
This instruction modifies the SEC_INFO field to allow an enclave to modify a page inside the enclave. The enclave requests the change to the page but then may accept the change to complete the process.
Instruction Inputs
Instruction Operation
Return:
If successful, the instruction sets the Z flag in the ELFAGS register to 0, otherwise the Z flag is set to one. The ENCLAVE_STATUS_MSR holds one of the following reasons:
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EACCEPT
Instruction Description
Software inside the enclave uses this instructions to accept changes to the SEC_INFO field. This allows the SMAP to be updated to a new page type.
Instruction Inputs
Instruction Operation
Return:
If successful, the instruction sets the Z flag in the ELFAGS register to 0, otherwise the Z flag is set to one. The ENCLAVE_STATUS_MSR holds one of the following reasons:
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EENTER
Instruction Description
The EENTER instruction transfers execution to an enclave. At the end of the instruction, the CPU is running in enclave mode at the IP specified in the TCS oENTRY or oHANDLER.
EENTER checks that TCS is a valid and available for entry. TCS and the corresponding SSA may be resident in memory for the instruction to proceed.
EENTER also checks the state machine to determine the type of entry and checks that only one logical processor is active inside a TCS at one time.
RFLAGS.TF has a slightly modified behavior on EENTER. RFLAGS.TF is stored into TCS.SAVE_TF and is then loaded from TCS.TF. A Debug Exception is then conditionally generated depending on the updated value of RFLAGS.TF.
If the enclave is in not Debug Mode, debug register DR7 is saved into TCS.DR7 and is cleared. Likewise for the IA32_DEBUGCTL MSR.
Instruction Inputs
Instruction Outputs
Instruction Operation
Flags Affected
None
Use of Prefixes
Lock: causes UD#
REP: causes UD#
Segment overrides: N/A
Operand Size: causes UD#
Address Size: ignored
RFLAGS.TF Behavior
The value of RFLAGS.TF at the start of execution of EENTER has no effect on a trap on completion of EENTER. Instead, the value of RFLAGS.TF that is loaded from the TCS determines whether or not a trap is taken.
DR7 Behavior
If the enclave is in not Debug Mode, debug register DR7 is saved into TCS.DR7 and is cleared.
IA32_DEBUG_CTL Behavior
If the enclave is in not Debug Mode, the IA32_DEBUG_CTL MSR is saved into TCS. DEBUG_CTL and is cleared.
Protected Mode Exceptions
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EEXIT
EEXIT exits to outside the enclave.
Instruction Description
EEXIT disables enclave mode and branches to the location specified in RBX.
No registers are affected by this instruction. If secrets are contained in any registers, it is responsibility of enclave software to clear those registers.
RFLAGS.TF has a slightly modified behavior on EEXIT. RFLAGS.TF is loaded from TCS.SAVE_TF. A Debug Exception is then conditionally generated depending on the updated value of RFLAGS.TF.
If the enclave is in not Debug Mode, debug register DR7 is loaded from TCS.DR7. This behavior and that of RFLAGS.TF is documented in more detail in???.
Instruction Inputs
Instruction Operation
Flags Affected
None
Use of Prefixes
Lock: causes UD#
REP: causes US#
Segment overrides: N/A
Operand Size: caused UD#
Address Size: ignored
RFLAGS.TF Behavior
The value of RFLAGS.TF at the start of execution of EEXIT has no effect on a trap on completion of EEXIT. Instead, the value of RFLAGS.TF that is loaded from the SSA determines whether or not a trap is taken.
DR7 Behavior
If the enclave is in not Debug Mode, debug register DR7 is loaded from TCS.DR7.
IA32_DEBUG_CTL Behavior
If the enclave is in not Debug Mode, the IA32_DEBUG_CTL MSR is loaded from TCS. DEBUG_CTL.
Protected Mode Exceptions
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EIRET
Instruction Description
The EIRET instruction resumes execution of an enclave that was interrupted due to an exception or interrupt using the machine state previously stored in the SSA.
EIRET checks that TCS is a valid and available for resumption. TCS and the corresponding SSA may be resident in memory for the instruction to proceed.
EIRET also checks the state machine to determine the type of entry and checks that only one logical processor is active inside a TCS at one time.
If RFLAGS.TF is set on EIRET, a Debug Exception will occur upon completion of the instruction, i.e. normal TF behavior. This exception will be reported as having occurred inside the enclave (in the usual SE-defined fashion), with no instructions having been executed inside. Since EIRET restores RFLAGS from the SSA, TF may become set at the end of EIRET. In this case, the TF will affect the following instruction; again, normal TF behavior.
Instruction Inputs
Instruction Operation
Flags Affected
None
Use of Prefixes
Lock: causes UD#
REP: causes UD#
Segment overrides: N/A
Operand Size: causes UD#
Address Size: ignored
RFLAGS.TF Behavior
If RFLAGS.TF is set at the start of the EIRET instruction, #DB will occur after completion. The exception will be reported at the RIP to which control would have been transferred had TF not been set. In effect, no forward progress within the enclave will occur.
As part of the normal operation of EIRET, RFLAGS is restored from the SSA copy. If the resulting TF is set, #DB will occur after execution of the target instruction inside the enclave.
These behaviors are consistent with those of the normal IA IRET instruction.
DR7 Behavior
DR7 is restored from the SSA copy that was previously save in the last interrupt or exception.
IA32_DEBUG_CTL Behavior
The IA32_DEBUG_CTL MSR is restored from the SSA copy that was previously save in the last interrupt or exception.
Protected Mode Exceptions
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EREPORT
The EREPORT instruction reports a measurement about the enclave contents
Instruction Description
EREPORT retrieves the enclave measurement registers, its capabilities and debug status (flags). All these values are protected using a symmetric message authentication code, which can be verified using the REPORT key. Enclaves which require the REPORT key may have the appropriate capability set in their SECS to retrieve it using the EGETKEY instruction.
The result of this instruction is deposited in the destination location, output_buffer_la.
Instruction Inputs
Instruction Operation
Flags Affected
None
Use of Prefixes
Lock: causes UD#
REP: causes US#
Segment overrides: N/A
Operand Size: caused UD#
Address Size: ignored
Protected Mode Exceptions
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
ERDMR
The ERDMR instruction reads the measurement register values out of the enclave SECS
Instruction Description
This instruction can only be executed from outside the enclave. If the SECS points to a valid SECS page then the instruction outputs the contents of the enclaves measurement registers to the address specified by output_buffer_la.
Instruction Inputs
Instruction Operation
Flags Affected
None
Use of Prefixes
Lock: causes UD#
REP: causes US#
Segment overrides: N/A
Operand Size: caused UD#
Address Size: ignored
Protected Mode Exceptions
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EGETKEY
Used by enclave code to return a particular key from the processor key hierarchy.
Instruction Description
The key required is specified using a KeyRequest structure, the address of which is provided as an input. This address may be naturally aligned.
The output is always a 256 bit data value. output_la needs to be naturally aligned for this value
Inputs
Instruction Operation
Flags Affected
None
Use of Prefixes
Lock: causes UD#
REP: causes US#
Segment overrides: N/A
Operand Size: caused UD#
Address Size: ignored
Protected Mode Exceptions
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
ERDTCSPTR
Instruction Description
The ERDTCSPTR instruction is used to read the current TCS linear address into RBX
Instruction Output
Instruction Output
Instruction Operation
Flags Affected
None
Use of Prefixes
Lock: causes UD#
REP: causes US#
Segment overrides: N/A
Operand Size: caused UD#
Address Size: ignored
Protected Mode Exceptions
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EDBGRD
Instruction Description
The EDBGRD instruction is used to read 8 bytes from debug enclave
Instruction Inputs
epc_la
dest
Instruction Operation
Flags Affected
None
Use of Prefixes
Lock: causes UD#
REP: causes UD#
Segment overrides: N/A
Operand Size: causes UD#
Address Size: ignored
Protected Mode Exceptions
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
EDBGWR
Instruction Description
The EDBGWR instruction is used to write 8 bytes to debug enclave page
Instruction Inputs
src
epc_la
Instruction Operation
Flags Affected
None
Use of Prefixes
TBD
Protected Mode Exceptions
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
ERDINFO
The ERDINFO instruction returns information about the contents of the enclave page cache
Instruction Description
If executed outside the enclave EREPORT reports the enclave measurement registers its capabilities and debug status (flags). All these values are protected using a symmetric message authentication code, which can be verified using the EVERIFYREPORT instruction.
The result of this instruction is deposited in the destination location, output_buffer_la.
Instruction Inputs
Instruction Operation
Flags Affected
RFLAGS.ZF
Use of Prefixes
Lock: causes UD#
REP: causes US#
Segment overrides: N/A
Operand Size: caused UD#
Address Size: ignored
Protected Mode Exceptions
Real Address Mode Exceptions
Virtual 8086 Mode Exceptions
Exits
This section provides pseudocode for exit processing. This code is invoked when there is an exit from the enclave which is not planned by the enclave code. Enclave execution is resumed at the place where it was stopped. Information needed to resume is saved on the outside stack. The architectural state of the processor is saved in the appropriate save area.
Operation
Acquire Reader Lock
RW locks enable logical processors to access shared resources and provide two modes in which threads can access a shared resource:
Flags Affected
None
Exceptions
None
Release Reader Lock
Subroutine Description
Releases lock acquired in shared mode.
Subroutine Inputs
volatile se_rw_lock_t*l
Subroutine Operation
Flags Affected
None
Exceptions
None
Acquire Writer Lock
Subroutine Description
Acquires lock in exclusive mode.
Subroutine Inputs
volatile se_rw_lock_t*l
Subroutine Operation
Flags Affected
None
Exceptions
None
Release Writer Lock
Subroutine Description
Releases lock acquired in exclusive mode.
Subroutine Inputs
volatile se_rw_lock_t*l
Subroutine Operation
Flags Affected
None
Exceptions
None
Xutranslate
Subroutine Description
This subroutine is in fact a hardware add by which the uCode exposes the PMH address-translation functionality to uCode. XUTRANSLATE is essentially a uOp that takes as input a PMH context, and a linear address, and produces the final physical address as the output. If the PMH encounters any fault conditions during the page table walk, those are reported to uCode. The exact details of this uOp are beyond the scope of this document.
Subroutine Inputs
TBD
Subroutine Operation
Flags Affected
None
Exceptions
None
DeriveKey
Subroutine Description
This subroutine is used to create a key by performing a CMAC operation over the DerivationBuffer with the identified key. DerivationBuffer needs to be a multiple of 128 bits.
Subroutine Inputs
pBaseKey—the base key to be used.
derivationBuffer—fixed size data field to be hashed.
Subroutine Operation
Flags Affected
None
Exceptions
None
This patent application is a continuation of U.S. patent application Ser. No. 13/527,547, filed Jun. 19, 2012, which is incorporated by reference, and which is a continuation-in-part of and claims priority to International Application No. PCT/US2009/069212, filed Dec. 22, 2009, entitled METHOD AND APPARATUS TO PROVIDE SECURE APPLICATION EXECUTION.
| Number | Name | Date | Kind |
|---|---|---|---|
| 7136488 | Hashimoto et al. | Nov 2006 | B2 |
| 8732371 | Berezecki | May 2014 | B2 |
| 9087200 | McKeen et al. | Jul 2015 | B2 |
| 20020101995 | Hashimoto et al. | Aug 2002 | A1 |
| 20020184046 | Kamada et al. | Dec 2002 | A1 |
| 20020194389 | Worley, Jr. | Dec 2002 | A1 |
| 20050105738 | Hashimoto et al. | May 2005 | A1 |
| 20060023884 | McKee | Feb 2006 | A1 |
| 20060075312 | Fischer et al. | Apr 2006 | A1 |
| 20070277223 | Datta et al. | Nov 2007 | A1 |
| 20080228649 | Zunke | Sep 2008 | A1 |
| 20090292894 | Henry | Nov 2009 | A1 |
| 20100318800 | Simon | Dec 2010 | A1 |
| 20120069995 | Matthews, Jr. | Mar 2012 | A1 |
| Number | Date | Country |
|---|---|---|
| 101116081 | Jan 2008 | CN |
| 101268650 | Sep 2008 | CN |
| 101454751 | Jun 2009 | CN |
| 1 278 114 | Jan 2003 | EP |
| 2002-232417 | Aug 2002 | JP |
| 2002-232417 | Aug 2002 | JP |
| 2002-353960 | Dec 2002 | JP |
| 2005-099984 | Apr 2005 | JP |
| 2001318787 | Nov 2011 | JP |
| 2012-508938 | Apr 2014 | JP |
| 10-2007-0118589 | Dec 2007 | KR |
| 10-2008-0074848 | Aug 2008 | KR |
| WO 2010057065 | May 2010 | WO |
| 2011078855 | Jun 2011 | WO |
| Entry |
|---|
| PCT International Search Report for PCT Counterpart Application No. PCT/US2009/069212, 2pgs. (dated Dec. 22, 2010). |
| PCT Written Opinion of the International Search Authority for PCT/US2009/069212, 6 pgs. (dated Dec. 22, 2010). |
| PCT International Preliminary Report on Patentability for PCT Application No. PCT/US2009/069212, 7pgs. (dated Jun. 26, 2012). |
| English translation of Preliminary Rejection for Korean Patent Application No. 2012-7016450, 3pgs. (dated Dec. 20, 2013). |
| First Office Action for Japanese Patent Application No. 2012-516046 with English translation, 9pgs. (dated Apr. 26, 2013). |
| Notice of Allowance for Japanese Patent Application No. 2012-516046 with English translation, 5pgs. (dated Nov. 19, 2013). |
| First Office Action for Chinese Patent Application No. 200980160114.X with English translation, 12 pgs. (dated Feb. 2, 2014). |
| Second Office Action for Chinese Patent Application No. 200980160114.X with English translation, 7pgs. (dated Oct. 11, 2014). |
| Notice of Allowance for Korean Patent Application No. 2012-7016450 with English translation, 7pgs. (dated Jul. 28, 2014). |
| Notice of Allowance for U.S. Appl. No. 13/527,547, 6pgs. (dated Mar. 13, 2015). |
| Non-Final Office Action for U.S. Appl. No. 13/527,547, 7pgs. (dated Oct. 21, 2014). |
| Third Office Action for Chinese Patent Application No. 200980160114.X, 7 pgs., (dated Apr. 9, 2015). |
| Fourth Office Action for Chinese Patent Application No. 200980160114.X with English translation, 7 pgs. (dated Dec. 15, 2015). |
| First Office Action for Japanese Patent Application No. 2013-262672 with English translation, 5 pgs. (dated Jun. 23, 2015). |
| First Office Action for German Patent Application No. 11 2009 005 466.8 with English translation, 8 pgs. dated Jan. 11, 2016). |
| Office action with English translation from Japanese Patent Application No. 2013-262672, dated Mar. 29, 2016, 6 pages. |
| Office action from UK Divisional Patent Application No. 1709341.0, dated Sep. 20, 2017, 4 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20130198853 A1 | Aug 2013 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13527547 | Jun 2012 | US |
| Child | 13802272 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/US2009/069212 | Dec 2009 | US |
| Child | 13527547 | US |
