The inventions generally relate to a symmetric inter-partition channel to stream data between partitions.
Modern computing systems may utilize multiple sets of processing resources, including perhaps processing cores, hyper-threads, and memory. A set of processing resources may be referred to herein as a “partition.” Some ideas of dealing with partitions that have been contemplated by the inventor include asymmetric inter-partition channels (IPCs) where each side of the IPC is different, and software implementations such as using an operating system (OS) and exposing an inter-partition bridge (IPB) using advanced configuration and power interface (ACPI) tables.
An asymmetric IPC implementation would likely require a special partition to support the device implementation side of the IPC. Regarding software implementations, previous patent application Ser. No. 11/241,247 entitled “EXPOSED SEQUESTERED PARTITION APPARATUS, SYSTEMS, AND METHODS” filed on Sep. 30, 2005 discusses an advanced configuration and power interface (ACPI) software exposure aspect of an inter-partition bridge (IPB). This application deals with exposing IPBs through ACPI software, and also discusses how a pseudo device (a system that believes that a device exists, where none really exists) is used to implement the IPB.
The inventions will be understood more fully from the detailed description given below and from the accompanying drawings of some embodiments of the inventions which, however, should not be taken to limit the inventions to the specific embodiments described, but are for explanation and understanding only.
Some embodiments of the inventions relate to a symmetric inter-partition channel to stream data between partitions.
In some embodiments, an inter-partition apparatus includes a set of registers to store direct memory access controls and to store an access control list visible to two or more operating environments separated by a partition, the set of registers including posted receive buffers and transmit pending buffers. A direct memory access device streams data in both directions between the two or more operating environments in response to the stored direct memory access controls and in response to the access control list using the posted receive buffers and the transmit pending buffers.
In some embodiments a system includes a processor and an inter-partition device. The inter-partition device streams data in both directions between two or more operating environments of the processor that are separated by a partition. The inter-partition device includes a set of registers and a direct memory access device. The set of registers stores direct memory access controls and also stores an access control list visible to the two or more operating environments, and includes posted receive buffers and transmit pending buffers. The direct memory access device streams data in both directions between the two or more operating environments in response to the stored direct memory access controls and in response to the access control list using the posted receive buffers and the transmit pending buffers.
In some embodiments inter-partition communication is implemented by storing in one or more registers direct memory access controls and an access control list visible to two or more operating environments separated by a partition. Data is streamed in both directions between the two or more operating environments in response to the stored direct memory access controls and in response to the access control list using direct memory access and using posted receive buffers and transmit pending buffers.
In some embodiments, computer platforms utilize partitioning of system resources including processor threads and/or processor cores. In some embodiments, communication is implemented in a controlled way between partitions, for example, relating to a platform resource layer (PRL) and/or many core implementations.
It is useful to partition a computer platform such as platform 100 such that one or more operating environments can run simultaneously. A useful partitioned platform typically exists if there are one or more computational cores on the platform (for example, a central processing unit or CPU in the platform includes one or more computational cores). Partitioning is accomplished by dividing up the platform resources (for example, one or more of Dynamic Random Access Memory or DRAM, peripheral devices, and/or CPU cores, etc.) and by reporting different portions of the divided resources to multiple operating environments. Each of the operating environments is centered on one or more computational cores (for example, CPU cores). In some embodiments, using hardware mechanisms, the resource division may be enforced such that one operating environment cannot damage the operation of another operating environment.
Each of the operating environments 102 and 104 illustrated in
The bottom half of
In some embodiments, hardware (for example, implemented in silicon) is provided that prevents operations in one operating environment (and/or core) to access memory allocated for the other operating environment (and/or core). This presents a problem when a controlled useful access is required by the other operating environment (and/or core). For example, it must be determined what happens if one operating environment (and/or processing environment and/or core) provides a service required by the other operating environment. In some embodiments, a mechanism is provided to bridge the two operating environments and provide a communication channel between the two environments. This communication channel appears as the same device to each of the two (or more) operating environments. In some embodiments, the communication channel provides a secure way (for example, point-to-point way) to stream information between partitions. In some embodiments the mechanism can be exposed as a platform device described in the ACPI (Advanced Configuration and Power Interface) tables presented to each operating environment. This mechanism appears as the same device to both operating environments. This “same device” feature can be described as a “symmetric” way to provide inter-partition channels according to some embodiments.
In some embodiments memory control unit 202 includes a host interface 204, a system memory direct memory interface A 206 (for example, a direct Rambus Direct Random Access Memory or direct RDRAM), a system memory direct memory interface B 208 (for example, a direct Rambus Direct Random Access Memory or direct RDRAM), an Advanced Graphics Port (AGP) interface 210, a HUB interface 212, clocks and reset circuitry 214, and a symmetric inter-partition bridge (IPB) logic block 216. In some embodiments
IPB block 216 is shown in an exploded view in
In some embodiments, device registers (for example, one or more IPC register sets 222 and/or one or more registers within IPC register sets 222) have status which allows both sides of the IPC (that is, two or more partitions) to synchronize a state of the data stream (for example, “device operational”, “device driver loaded”, etc.) The device implements a direct memory access (DMA) engine (for example, via DMA channel 226 and/or DMA channel 228) that allows for the streaming of data through the IPC. The DMA engine is subject to an access control list (ACL) of physical memory pages submitted by a trusted configuration entity on each side (each partition) of the IPC, for example. The trusted configuration entity on each side of the IPC sets up a table of pages (on the respective side) that are accessible on their side through the IPC. The DMA channel (for example, implemented in silicon) is part of the disclosure, and checks the tables to make sure that the DMA operation is permitted. If the DMA operation is not permitted, then DMA does not occur.
As illustrated in
In some embodiments, one or more of the “Partition A” to “Partition B” DMA register blocks 342 include a “Partition A” to “Partition B” Buffer Physical Address (A Side) register 352, a “Partition A” to “Partition B” Buffer size (A side) register 354, a command/status register 356, a “Partition A” to “Partition B” Buffer Physical Address (B Side) register 358, and/or a “Partition A” to “Partition B” Buffer size (B side) register 360. In some embodiments, one or more of the “Partition B” to “Partition A” DMA register blocks 344 include a “Partition B” to “Partition A” Buffer Physical Address (A Side) register 362, a “Partition B” to “Partition A” Buffer size (A side) register 364, a command/status register 366, a “Partition B” to “Partition A” Buffer Physical Address (B Side) register 368, and/or a “Partition B” to “Partition A” Buffer size (B side) register 370.
In some embodiments the “Partition A” to “Partition B” DMA blocks and/or the “Partition B” to “Partition A” DMA blocks are double ringed. That is, addresses for both the transmitting side and the receiving side are visible in the same structure in the registers of the device. This makes is possible for a symmetrical and useful device. Further, the DMA access control list is accessible in the registers so that, for example, the DMA engine is able to check the ACL registers prior to DMA operations.
As discussed above, having multiple partitions is becoming very popular in the computer industry. Some embodiments provide a very natural model to stream data across partitions. In some embodiments, communication is provided across partitions in a manner such that both sides can treat the communication channel in the same way. It is not necessary to distinguish between the consumer of the data and the provider of the data, as would be in the case of using an asymmetric inter-partition channel device. Implementation in hardware (for example, in silicon) is a natural embodiment for such a symmetric IPC device.
In some embodiments, symmetric IPC solves a general partitioning inter-partition channel problem. In some embodiments, the same sample driver core code can be used on either side of the inter-partition bridge, providing a very clean and easily implemented solution. There is no requirement that the bridged partitions have specialized roles in view of the symmetric nature of the IPC.
In some embodiments, two or more partitions see the same device from different sides. In some embodiments, a double ring is used (posted receive buffers and transmit pending buffers). The double ring is located in the registers, allowing for out of order return of receive buffers, and performs streaming such that silicon and software driver code may be reused. This results in a symmetric nature for an IPC, since the same software driver core can be used from both sides.
In some embodiments, IPC devices are implemented in a memory controller block (for example, that is physically close to the CPU and can make the device perform in a fast manner). In some embodiments, device discovery does not involve, for example, PCI enumeration which adds complexity to the silicon design. Instead, in some embodiments, ACPI may be used for device discovery.
In some embodiments, the streaming model looks like a mail box when the number of buffers is reduced to one. In this manner, the design decomposes to a mailbox model when the number of buffers are limited, providing the driver writer with flexibility in the way that the device functions.
IN some embodiments, an IPB device and/or an IPC device are implemented in silicon, and page table entries where DMA can be performed on that data are also in registers accessed only by a trusted entity. If the page lists are empty, the silicon does not perform DMA. This results in a secure and fast device.
Some embodiments apply to a partitioned computer platform having more than one CPU core and/or hardware thread, where data is shared through a partition boundary.
In some embodiments, symmetric IPC devices are exposed to the operating system (OS) so that device drivers can be loaded. In some embodiments, a stream of data may be produced in one partition and consumed in another partition.
In some embodiments, a symmetric IPC device (and/or symmetric IPB device) are implemented in silicon and exposed in registers that are included in a chip set (for example, in a memory controller hub of a chipset).
In some embodiments, a symmetric IPC device such as IPB device 216 in
In some embodiments, an IPC device is a minimalist robust DMA streaming interface. In some embodiments, the streaming interface involves two or more DMA channels (for example, for full duplex streams). In some embodiments, the two or more streams are implemented using a double ring of memory pointers for each direction. The use of a double ring (that is, for example, posted receive buffers and transmit pending buffers) located in the registers allows for out of order return of receive buffers, and provides all properties of a great streaming device. Since this mechanism is the same in each direction, it is possible to reuse the silicon and drivers for each direction, providing the symmetric nature of the IPC. Further, the same software driver core can be used from both sides.
In some embodiments, the DMA engine (for example, DMA hardware engine) checks an access control list (ACL) included within the IPC registers prior to DMA operations. A page list is write accessible in configuration context, and is implemented in ACL registers. In some embodiments only a small number of ACL registers is necessary. In some embodiments, each operating environment builds the access control list (ACL) and registers the list to the IPB device. Otherwise, the DMA engine does not perform any DMA operations through the IPC.
In some embodiments, the configuration of the IPC is done in a privileged mode on both sides of the IPC (that is, in two or more operating environments or partitions). In some embodiments, two or more partitions see the same device from different sides.
In some embodiments an array of symmetric IPCs are implemented at a memory complex. In some embodiments, an IPC hardware device is implemented in silicon for each IPC. In some embodiments, the IPC device is identified to two or more operating environments using ACPI tables. In some embodiments, synchronization between two or more sides of an IPC is maintained through status bits available in registers visible to two or more operating environments.
In some embodiments, a hardware inter-partition channel (IPC) has a performance advantage over virtualization techniques used to perform similar functions. In some embodiments, a hardware IPC is used by virtualization technology to perform inter-partition bridging and channeling functions.
In some embodiments, a symmetric inter-partition channel (IPC) is used for general partitioning in a system where two or more operating environments operate on two or more sides of an inter-partition channel (IPC). In some embodiments, two or more operating environments load a driver for an inter-partition channel in a manner that is natural for the two or more operating environments, with no driver loading temporal order, for example.
In some embodiments IPC and DMA operations are implemented in silicon, and the DMA operations are subject to an access control list (ACL) and the configuration of an IPC is performed only in a privileged mode, thus providing a secured IPC.
In some embodiments, core driver code requires interfacing an IPC device in a manner that is common for all drivers, and the core driver code can be given out as a driver reference. Giving out the driver code does not compromise security because security is derived from the access control list, which is populated by the privileged partition manager, for example.
In some embodiments, data copies are performed using DMA channels rather than by using CPU cores.
The terms “inter-partition bridge” (or “IPB”) and “inter-partition channel” (or “IPC”) are used herein in very similar ways. It is noted that in cases where an IPB has only one IPC, these terms are interchangeable with each other, and have the same meaning. However, an IPB can have one or more IPC channel. For example, in some embodiments (for example, as illustrated in
Although some-embodiments have been described herein as being implemented in hardware and/or in silicon, according to some embodiments these particular implementations may not be required.
Although some embodiments have been described herein as two or more operating environments that correspond to two or more different cores in a single processor package and/or that operate on two or more hardware threads in a single processor core, in some embodiments, the two or more different cores and/or the two or more hardware threads may not be contained in a single processor package. They may be contained in two or more different processor packages, for example. In some embodiments each of the different cores and/or hardware threads may be contained in a processor package that is different than all the other different cores and/or hardware threads, for example, or in some embodiments, for example, some of the different cores and/or different hardware threads are contained in a first processor package and others of the different cores and/or different hardware threads are contained in a second processor package.
In some embodiments, the IPC is implemented within an MCH. In some embodiments, more than one processor may be coupled with the same MCH, and more than one processor package containing multiple cores can use the same IPC, for example.
Although some embodiments have been described in reference to particular implementations, other implementations are possible according to some embodiments. Additionally, the arrangement and/or order of circuit elements or other features illustrated in the drawings and/or described herein need not be arranged in the particular way illustrated and described. Many other arrangements are possible according to some embodiments.
In each system shown in a figure, the elements in some cases may each have a same reference number or a different reference number to suggest that the elements represented could be different and/or similar. However, an element may be flexible enough to have different implementations and work with some or all of the systems shown or described herein. The various elements shown in the figures may be the same or different. Which one is referred to as a first element and which is called a second element is arbitrary.
In the description and claims, the terms “coupled” and “connected,” along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. Rather, in particular embodiments, “connected” may be used to indicate that two or more elements are in direct physical or electrical contact with each other. “Coupled” may mean that two or more elements are in direct physical or electrical contact. However, “coupled” may also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.
An algorithm is here, and generally, considered to be a self-consistent sequence of acts or operations leading to a desired result. These include physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers or the like. It should be understood, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities.
Some embodiments may be implemented in one or a combination of hardware, firmware, and software. Some embodiments may also be implemented as instructions stored on a machine-readable medium, which may be read and executed by a computing platform to perform the operations described herein. A machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer). For example, a machine-readable medium may include read only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, the interfaces that transmit and/or receive signals, etc.), and others.
An embodiment is an implementation or example of the inventions. Reference in the specification to “an embodiment,” “one embodiment,” “some embodiments,” or “other embodiments” means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least some embodiments, but not necessarily all embodiments, of the inventions. The various appearances “an embodiment,” “one embodiment,” or “some embodiments” are not necessarily all referring to the same embodiments.
Not all components, features, structures, characteristics, etc. described and illustrated herein need be included in a particular embodiment or embodiments. If the specification states a component, feature, structure, or characteristic “may”, “might”, “can” or “could” be included, for example, that particular component, feature, structure, or characteristic is not required to be included. If the specification or claim refers to “a” or “an” element, that does not mean there is only one of the element. If the specification or claims refer to “an additional” element, that does not preclude there being more than one of the additional element.
Although flow diagrams and/or state diagrams may have been used herein to describe embodiments, the inventions are not limited to those diagrams or to corresponding descriptions herein. For example, flow need not move through each illustrated box or state or in exactly the same order as illustrated and described herein.
The inventions are not restricted to the particular details listed herein. Indeed, those skilled in the art having the benefit of this disclosure will appreciate that many other variations from the foregoing description and drawings may be made within the scope of the present inventions. Accordingly, it is the following claims including any amendments thereto that define the scope of the inventions.
This application is related to U.S. patent application Ser. No. 11/027,253 filed on Dec. 30, 2004. This application is also related to U.S. patent application Ser. No. 11/241,247 filed on Sep. 30, 2005.