Patent Application
20180331900
Embodiments generally relate to data processing systems. More particularly, embodiments relate to a reconfigurable device manager.
A reconfigurable device such as a field programmable gate array (FPGA) may be reconfigured to provide different hardware functionality. A data center may include and/or utilize a number of FPGA devices. Some data centers may permit a FPGA device to be reconfigured at the request of an end user. This practice is sometimes referred to as FPGA-as-a-Service (FaaS).
The various advantages of the embodiments will become apparent to one skilled in the art by reading the following specification and appended claims, and by referencing the following drawings, in which:
Turning now to
Embodiments of each of the above processor 11, persistent storage media 12, reconfigurable device 13, device manager 14, request processor 15, configuration controller 16, pseudo-device manager 17, and other system components may be implemented in hardware, software, or any suitable combination thereof. For example, hardware implementations may include configurable logic such as, for example, programmable logic arrays (PLAs), FPGAs, complex programmable logic devices (CPLDs), or fixed-functionality logic hardware using circuit technology such as, for example, application specific integrated circuit (ASIC), complementary metal oxide semiconductor (CMOS) or transistor-transistor logic (TTL) technology, or any combination thereof.
Alternatively, or additionally, all or portions of these components may be implemented in one or more modules as a set of logic instructions stored in a machine- or computer-readable storage medium such as random access memory (RAM), read only memory (ROM), programmable ROM (PROM), firmware, flash memory, etc., to be executed by a processor or computing device. For example, computer program code to carry out the operations of the components may be written in any combination of one or more operating system (OS) applicable/appropriate programming languages, including an object-oriented programming language such as PYTHON, PERL, JAVA, SMALLTALK, C++, C# or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. For example, the persistent storage media 12 may store a set of instructions which when executed by the processor 11 cause the system 10 to implement one or more components or aspects of the system 10 (e.g., the device manager 14, the request processor 15, the configuration controller 16, the pseudo-device manager 17, etc.).
Turning now to
In some embodiments, the pseudo-device manager 23 may be configured to create the pseudo device in compliance with an OS. For example, the pseudo-device manager 23 may assign a device class to the pseudo device that corresponds to an OS driver for the device class. In any of the foregoing examples, the request processor 21 may include a host interface. Alternatively, or additionally, in some embodiments the request processor 21 may include a network interface. In a virtualization environment, for example, the request processor 21 may comprise a virtualized interface and the configuration controller may comprise a virtualized configuration controller. For the virtualization environment, for example, some embodiments may further include a mapping layer to map function interface resources to the virtualized interface.
Embodiments of each of the above request processor 21, configuration controller 22, pseudo-device manager 23, and other components of the apparatus 20 may be implemented in hardware, software, or any combination thereof. For example, hardware implementations may include configurable logic such as, for example, PLAs, FPGAs, CPLDs, or fixed-functionality logic hardware using circuit technology such as, for example, ASIC, CMOS, or TTL technology, or any combination thereof.
Alternatively, or additionally, these components may be implemented in one or more modules as a set of logic instructions stored in a machine- or computer-readable storage medium such as RAM, ROM, PROM, firmware, flash memory, etc., to be executed by a processor or computing device. For example, computer program code to carry out the operations of the components may be written in any combination of one or more OS applicable/appropriate programming languages, including an object-oriented programming language such as PYTHON, PERL, JAVA, SMALLTALK, C++, C# or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages.
Turning now to
Some embodiments of the method 30 may further include creating the pseudo device in compliance with an OS at block 36, and assigning a device class to the pseudo device that corresponds to an OS driver for the device class at block 37. Some embodiments of the method 30 may also include processing one or more of a host interface request and/or a network interface request for the reconfiguration of the reconfigurable device at block 38. Some embodiments may also include processing a virtualized interface request for the reconfiguration of the reconfigurable device at block 39, and mapping function interface resources to the virtualized interface at block 40.
Embodiments of the method 30 may be implemented in a system, apparatus, processor, reconfigurable device, etc., for example, such as those described herein. More particularly, hardware implementations of the method 30 may include configurable logic such as, for example, PLAs, FPGAs, CPLDs, or in fixed-functionality logic hardware using circuit technology such as, for example, ASIC, CMOS, or TTL technology, or any combination thereof. Alternatively, or additionally, the method 30 may be implemented in one or more modules as a set of logic instructions stored in a machine- or computer-readable storage medium such as RAM, ROM, PROM, firmware, flash memory, etc., to be executed by a processor or computing device. For example, computer program code to carry out the operations of the components may be written in any combination of one or more OS applicable/appropriate programming languages, including an object-oriented programming language such as PYTHON, PERL, JAVA, SMALLTALK, C++, C# or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. For example, the method 30 may be implemented on a non-transitory computer readable medium as described in connection with Examples 19 to 25 below. For example, embodiments or portions of the method 30 may be implemented in applications (e.g., through an application programming interface (API)) or driver software running on an OS.
Some embodiments may advantageously provide for efficiently managing dynamically reconfigurable hardware. Reconfigurable devices such as FPGAs may be important for various computing scenarios including, for example, in a datacenter or for various internet-of-things (IoT) usages. An example value proposition of such devices is that they may be dynamically reconfigured during system operation. At one instance, the device might have a compression persona while at another, it may have an encryption persona, a networking persona, a machine-learning persona, etc. Managing such changes may fall into the system software domain but conventional OS driver stacks may have limited capability to manage a dynamically reconfigurable device during runtime and/or may have no efficient mechanism to tear-down and build-up a driver stack in response to a persona change in such a reconfigurable device.
Advantageously, some embodiments may provide a hardware/software technique for handling such dynamically reconfigurable devices in both non-virtualized and virtualized system environments. For example, some embodiments may dynamically create and destroy virtual devices that represent individual functionality within a reconfigurable device (e.g., an FPGA). A reconfigurable device may be dynamically morphed to appear as a completely new kind of device exposing a completely different functionality, essentially becoming a completely new set of resources. In a heterogeneous system configuration, there may be many such reconfigurable devices to provide, for example, compute and/or input/output (IO) acceleration in addition to processors or central processor units (CPUs). In accordance with some embodiments, virtual devices may be created to represent dynamically configured physical resources, such as memory-mapped IO (MMIO) regions, interrupts, and system memory, in a manner that can be dynamically handled by system and/or virtualization software, while also ensuring an efficient CPU-to-reconfigurable device communication.
One approach to handle a reconfigured device in a system may require a reboot of the system. This approach may have significant drawbacks in the amount of time required to reboot and halting other applications running on the system. Reboot may not be practical in various scenarios such as cloud data centers, and some IoT usages (e.g., self-driving automobiles). Another approach may involve simulating a hot plug event for the reconfigured device. The simulated hot-plug approach may require additional hardware support to simulate the hot-plug sequence (e.g., by requiring electrical isolation from the device interconnect), may not support multiple functions in a single package (e.g., or may interrupt current functions to add a new function), and may be unreliable due to the complexity of hot-plug mechanisms. Another approach may involve hiding the persona change from the OS. Hiding the persona change from the OS, however, may preclude the applicability of the reconfigured device to portions of the OS that do not have visibility into the device (e.g., kernel mode usage scenarios). All the foregoing approaches introduce various compromises that may reduce the value proposition of the reconfigurable device.
In accordance with some embodiments, functions or portions of a reconfigurable device may be represented as virtual devices that may be dynamically instantiated and torn down without requiring a reboot or a hot-plug event. For example, virtual devices may be created in a form that is compliant with and known to the existing OS software. The OS-compliant virtual devices may allow OS and/or other kernel-mode software to utilize services implemented by the virtual device along with traditional application or user-mode usages.
In some embodiments, attributes or characteristics may be associated with each virtual device that may allow a user or system designer to override the device class of the underlying physical device such that the virtual device may appear to be a different device class. Having an appropriate device class assigned to the virtual device may allow existing class or device drivers to load without modification and utilize the virtual device.
For example, using this technique a network interface card (NIC) functionality may be implemented on an FPGA device while utilizing existing NIC drivers and OS infrastructure. In another example, the ability to load existing drivers may also enable software developers to implement pre-silicon validation of driver, software stack and register transfer logic (RTL) for a new device by initially implementing the new device on an FPGA. When the silicon for the new device (e.g., an ASIC) is ready, the drivers may be able to be deployed having already been debugged.
Turning now to
In some embodiments, the static functions may always be present as part of the reconfigurable device 42, whereas the dynamic functions may be created as new functions and/or removed by the configuration controller 44. Each of the static and dynamic functions may include various device interface resources, such as but not limited to MMIO regions, interrupts, resets, direct memory access (DMA) channels, etc., through which host/network software may communicate with or otherwise utilize the functions of the reconfigurable device 42. For example, MMIO regions for the functions of the reconfigurable device 42 may be scattered/partitioned (e.g., if required) to allow system software to map privileged MMIO to privileged software and expose nonprivileged resources directly to an application domain for fast-path communication. In some embodiments, existing mechanisms of page-table mapping may be used in connection with the reconfigurable device 42.
Some embodiments may advantageously partially reconfigure the reconfigurable device 42 to dynamically reconfigure one or more reconfigurable slots and dynamically instantiate the corresponding software stack for use by the requester. The persona of the reconfigurable slots may advantageously be respectively changed by instantiation and teardown. For example, the software stack may be torn down once the requester is done with the persona. The software stacks may also be torn down when another request displaces the current persona. In some embodiments, the reconfigurable device 42 may be part of a multi-chip package (MCP) such as an INTEL XEON+FPGA MCP. In some embodiments, the reconfigurable device 42 may be part of a Discrete Configurable Platform (DCP).
Turning now to
For example, an embodiment may include a reconfigurable device with two reconfigurable slots in an initial state without any particular persona or functionality programmed. The management application 55 may request the physical device driver 53 to program a compression function with three independent engines in the first reconfigurable slot. The physical device driver 53 may reprogram the physical hardware, query the capabilities, and based on the queried capabilities, the driver 53 may then create three virtual devices on the virtual bus. Each of the virtual devices may respectively represent one of the three compression engines programmed in the first reconfigurable slot of the reconfigurable device. These virtual devices may be enumerated on the virtual bus 54 and may be represented with virtual device objects (e.g., objects V1 through VN) for which system software may load various independent software stacks (e.g., kernel-mode (KM) driver 57, user-mode (UM) driver 58, existing device or class driver 59, etc.).
Subsequently, the management application 55 and/or host applications 51 may request the device driver 53 to reprogram the first reconfigurable slot with an encryption function containing two independent engines. The physical device driver 53 may first tear down the existing virtual devices and device drivers associated with that slot, reprogram the slot, and then create two new virtual devices representing the two encryption engines. In some embodiments, the physical device driver 53 may be responsible for managing the life cycle of functions in the reconfigurable device. For example, the physical device driver 53 may interact with the hardware to initiate reconfiguration activities, and may also interact with the OS to facilitate the creation and destruction of the virtual device objects that correspond with functions in the hardware.
In some embodiments, the physical device driver 53 may create or request creation of child virtual device objects representing respective functions in the reconfigurable device. Each of these virtual device objects may be augmented with separate MMIO, interrupts, or other capabilities/resources providing an interface to a higher-level software stack as if they were separate physical devices. The virtual device objects may also provide a mechanism (e.g., through APIs) to interact with resources that cannot be directly assigned, such as shared interrupts or MMIO regions. Because the physical driver 53 may distribute resources (e.g., MMIO, interrupts etc.) among the virtual devices, the driver 53 may assure software-level isolation between the virtual devices as long the resources are not interrelated. For example, the driver 53 may provide mapping and APIs such that interface resource isolation is maintained between the virtual devices as needed.
In some embodiments, the virtual device objects may be created and/or augmented with resources in a form that is known to the OS. Advantageously, an existing driver stack may be loaded and/or existing software may be used if the MMIO and interrupt layout for the virtual device object is compatible with the real physical device for which the original software was built. For example, an off-the-shelf gigabit Ethernet controller may be programmed in an FPGA and the created virtual device may expose the same hardware resources (MMIO, interrupts, ports etc.) as the real device. Advantageously, the virtual device may be a pseudo device that appears to be a real device. Advantageously, some embodiments may also expose new functionality that may be exposed through an entirely new software stack. In some embodiments, both a legacy function and a new function may be supported by the same reconfigurable device at the same time (e.g., the reconfigurable device may support any of a number of programmed functions with appropriate corresponding virtual devices). In some embodiments, a runtime application may be loaded to map interface resources appropriately to the user space, when feasible, or to provide APIs to allow access to resources, or alternatively, when a direct map is not desirable
Turning now to
Turning now to
In some embodiments, the physical device driver 77 on the host portion 71 may map device resources or functions to virtualized host channels. One example of resource mapping may include the host mapping the ability to reconfigure slot RS1, in
In some embodiments, the pseudo-device manager 132c may create the pseudo device in compliance with an OS. For example, the pseudo-device manager 132c may assign a device class to the pseudo device that corresponds to an OS driver for the device class. In any of the foregoing examples, the request processor 132a may comprise one or more of a host interface and/or a network interface. In a virtualization environment, for example, the request processor 132a may comprise a virtualized interface and the configuration controller 132b may comprise a virtualized configuration controller. For the virtualization environment, for example, the apparatus 132 may include a mapping layer to map function interface resources to the virtualized interface.
Turning now to
The processor core 200 is shown including execution logic 250 having a set of execution units 255-1 through 255-N. Some embodiments may include a number of execution units dedicated to specific functions or sets of functions. Other embodiments may include only one execution unit or one execution unit that can perform a particular function. The illustrated execution logic 250 performs the operations specified by code instructions.
After completion of execution of the operations specified by the code instructions, back end logic 260 retires the instructions of the code 213. In one embodiment, the processor core 200 allows out of order execution but requires in order retirement of instructions. Retirement logic 265 may take a variety of forms as known to those of skill in the art (e.g., re-order buffers or the like). In this manner, the processor core 200 is transformed during execution of the code 213, at least in terms of the output generated by the decoder, the hardware registers and tables utilized by the register renaming logic 225, and any registers (not shown) modified by the execution logic 250.
Although not illustrated in
Referring now to
The system 1000 is illustrated as a point-to-point interconnect system, wherein the first processing element 1070 and the second processing element 1080 are coupled via a point-to-point interconnect 1050. It should be understood that any or all of the interconnects illustrated in
As shown in
Each processing element 1070, 1080 may include at least one shared cache 1896a, 1896b (e.g., static random access memory/SRAM). The shared cache 1896a, 1896b may store data (e.g., objects, instructions) that are utilized by one or more components of the processor, such as the cores 1074a, 1074b and 1084a, 1084b, respectively. For example, the shared cache 1896a, 1896b may locally cache data stored in a memory 1032, 1034 for faster access by components of the processor. In one or more embodiments, the shared cache 1896a, 1896b may include one or more mid-level caches, such as level 2 (L2), level 3 (L3), level 4 (L4), or other levels of cache, a last level cache (LLC), and/or combinations thereof.
While shown with only two processing elements 1070, 1080, it is to be understood that the scope of the embodiments are not so limited. In other embodiments, one or more additional processing elements may be present in a given processor. Alternatively, one or more of processing elements 1070, 1080 may be an element other than a processor, such as an accelerator or a field programmable gate array. For example, additional processing element(s) may include additional processors(s) that are the same as a first processor 1070, additional processor(s) that are heterogeneous or asymmetric to processor a first processor 1070, accelerators (such as, e.g., graphics accelerators or digital signal processing (DSP) units), field programmable gate arrays, or any other processing element. There can be a variety of differences between the processing elements 1070, 1080 in terms of a spectrum of metrics of merit including architectural, micro architectural, thermal, power consumption characteristics, and the like. These differences may effectively manifest themselves as asymmetry and heterogeneity amongst the processing elements 1070, 1080. For at least one embodiment, the various processing elements 1070, 1080 may reside in the same die package.
The first processing element 1070 may further include memory controller logic (MC) 1072 and point-to-point (P-P) interfaces 1076 and 1078. Similarly, the second processing element 1080 may include a MC 1082 and P-P interfaces 1086 and 1088. As shown in
The first processing element 1070 and the second processing element 1080 may be coupled to an I/O subsystem 1090 via P-P interconnects 10761086, respectively. As shown in
In turn, I/O subsystem 1090 may be coupled to a first bus 1016 via an interface 1096. In one embodiment, the first bus 1016 may be a Peripheral Component Interconnect (PCI) bus, or a bus such as a PCI Express bus or another third generation I/O interconnect bus, although the scope of the embodiments are not so limited.
As shown in
Note that other embodiments are contemplated. For example, instead of the point-to-point architecture of
Example 1 may include an electronic processing system, comprising a processor, persistent storage media communicatively coupled to the processor, a reconfigurable device communicatively coupled to the processor, and a device manager communicatively coupled to the reconfigurable device to dynamically reconfigure the reconfigurable device at runtime, the reconfigurable device manager including a request processor to process a request for a reconfiguration of the reconfigurable device, a configuration controller to reconfigure the reconfigurable device based on the request, and a pseudo-device manager to create a pseudo device based on the request which corresponds to a functionality of the reconfiguration.
Example 2 may include the system of Example 1, wherein the pseudo-device manager is further to create a virtual bus to access the functionality of the pseudo device.
Example 3 may include the system of any of Examples 1 to 2, wherein the request processor comprises one or more of a host interface and a network interface.
Example 4 may include a device manager apparatus, comprising a request processor to process a request for a reconfiguration of a reconfigurable device, a configuration controller communicatively coupled to the request processor to reconfigure the reconfigurable device based on the request, and a pseudo-device manager communicatively coupled to the request processor to create a pseudo device based on the request which corresponds to a functionality of the reconfiguration.
Example 5 may include the apparatus of Example 4, wherein the pseudo-device manager is further to destroy a previously created pseudo device based on the request.
Example 6 may include the apparatus of Example 4, wherein the pseudo-device manager is further to create a virtual bus to access the functionality of the pseudo device.
Example 7 may include the apparatus of Example 6, wherein the pseudo-device manager is further to create the pseudo device in compliance with an operating system.
Example 8 may include the apparatus of Example 7, wherein the pseudo-device manager is further to assign a device class to the pseudo device that corresponds to an operating system driver for the device class.
Example 9 may include the apparatus of any of Examples 4 to 8, wherein the request processor comprises a host interface.
Example 10 may include the apparatus of any of Examples 4 to 8, wherein the request processor comprises a network interface.
Example 11 may include the apparatus of any of Examples 4 to 8, wherein the request processor comprises a virtualized interface and wherein the configuration controller comprises a virtualized configuration controller, and a mapping layer to map function interface resources to the virtualized interface.
Example 12 may include a method of managing a device, comprising processing a request for a reconfiguration of a reconfigurable device, reconfiguring the reconfigurable device based on the request, and creating a pseudo device based on the request which corresponds to a functionality of the reconfiguration.
Example 13 may include the method of Example 12, further comprising destroying a previously created pseudo device based on the request.
Example 14 may include the method of Example 12, further comprising creating a virtual bus to access the functionality of the pseudo device.
Example 15 may include the method of Example 14, further comprising creating the pseudo device in compliance with an operating system.
Example 16 may include the method of Example 15, further comprising assigning a device class to the pseudo device that corresponds to an operating system driver for the device class.
Example 17 may include the method of any of Examples 12 to 16, further comprising processing one or more of a host interface request and a network interface request for the reconfiguration of the reconfigurable device.
Example 18 may include the method of any of Examples 12 to 16, further comprising processing a virtualized interface request for the reconfiguration of the reconfigurable device, and mapping function interface resources to the virtualized interface.
Example 19 may include at least one non-transitory computer readable medium, comprising a set of instructions, which when executed by a computing device, cause the computing device to process a request for a reconfiguration of a reconfigurable device, reconfigure the reconfigurable device based on the request, and create a pseudo device based on the request which corresponds to a functionality of the reconfiguration.
Example 20 may include the at least one non-transitory computer readable medium of Example 19, comprising a further set of instructions, which when executed by a computing device, cause the computing device to destroy a previously created pseudo device based on the request.
Example 21 may include the at least one non-transitory computer readable medium of Example 19, comprising a further set of instructions, which when executed by a computing device, cause the computing device to create a virtual bus to access the functionality of the pseudo device.
Example 22 may include the at least one non-transitory computer readable medium of Example 21, comprising a further set of instructions, which when executed by a computing device, cause the computing device to create the pseudo device in compliance with an operating system.
Example 23 may include the at least one non-transitory computer readable medium of Example 22, comprising a further set of instructions, which when executed by a computing device, cause the computing device to assign a device class to the pseudo device that corresponds to an operating system driver for the device class.
Example 24 may include the at least one non-transitory computer readable medium of any of Examples 19 to 23, comprising a further set of instructions, which when executed by a computing device, cause the computing device to process one or more of a host interface request and a network interface request for the reconfiguration of the reconfigurable device.
Example 25 may include the at least one non-transitory computer readable medium of any of Examples 19 to 23, comprising a further set of instructions, which when executed by a computing device, cause the computing device to process a virtualized interface request for the reconfiguration of the reconfigurable device, and map function interface resources to the virtualized interface.
Example 26 may include a device manager apparatus, comprising means for processing a request for a reconfiguration of a reconfigurable device, means for reconfiguring the reconfigurable device based on the request, and means for creating a pseudo device based on the request which corresponds to a functionality of the reconfiguration.
Example 27 may include the apparatus of Example 26, further comprising means for destroying a previously created pseudo device based on the request.
Example 28 may include the apparatus of Example 26, further comprising means for creating a virtual bus to access the functionality of the pseudo device.
Example 29 may include the apparatus of Example 28, further comprising means for creating the pseudo device in compliance with an operating system.
Example 30 may include the apparatus of Example 29, further comprising means for assigning a device class to the pseudo device that corresponds to an operating system driver for the device class.
Example 31 may include the apparatus of any of Examples 26 to 30, further comprising means for processing one or more of a host interface request and a network interface request for the reconfiguration of the reconfigurable device.
Example 32 may include the apparatus of any of Examples 26 to 30, further comprising means for processing a virtualized interface request for the reconfiguration of the reconfigurable device, and means for mapping function interface resources to the virtualized interface.
Embodiments are applicable for use with all types of semiconductor integrated circuit (“IC”) chips. Examples of these IC chips include but are not limited to processors, controllers, chipset components, programmable logic arrays (PLAs), memory chips, network chips, systems on chip (SoCs), SSD/NAND controller ASICs, and the like. In addition, in some of the drawings, signal conductor lines are represented with lines. Some may be different, to indicate more constituent signal paths, have a number label, to indicate a number of constituent signal paths, and/or have arrows at one or more ends, to indicate primary information flow direction. This, however, should not be construed in a limiting manner. Rather, such added detail may be used in connection with one or more exemplary embodiments to facilitate easier understanding of a circuit. Any represented signal lines, whether or not having additional information, may actually comprise one or more signals that may travel in multiple directions and may be implemented with any suitable type of signal scheme, e.g., digital or analog lines implemented with differential pairs, optical fiber lines, and/or single-ended lines.
Example sizes/models/values/ranges may have been given, although embodiments are not limited to the same. As manufacturing techniques (e.g., photolithography) mature over time, it is expected that devices of smaller size could be manufactured. In addition, well known power/ground connections to IC chips and other components may or may not be shown within the figures, for simplicity of illustration and discussion, and so as not to obscure certain aspects of the embodiments. Further, arrangements may be shown in block diagram form in order to avoid obscuring embodiments, and also in view of the fact that specifics with respect to implementation of such block diagram arrangements are highly dependent upon the platform within which the embodiment is to be implemented, i.e., such specifics should be well within purview of one skilled in the art. Where specific details (e.g., circuits) are set forth in order to describe example embodiments, it should be apparent to one skilled in the art that embodiments can be practiced without, or with variation of, these specific details. The description is thus to be regarded as illustrative instead of limiting.
The term “coupled” may be used herein to refer to any type of relationship, direct or indirect, between the components in question, and may apply to electrical, mechanical, fluid, optical, electromagnetic, electromechanical or other connections. In addition, the terms “first”, “second”, etc. may be used herein only to facilitate discussion, and carry no particular temporal or chronological significance unless otherwise indicated.
As used in this application and in the claims, a list of items joined by the term “one or more of” may mean any combination of the listed terms. For example, the phrases “one or more of A, B, and C” may mean A; B; C; A and B; A and C; B and C; or A, B and C.
Those skilled in the art will appreciate from the foregoing description that the broad techniques of the embodiments can be implemented in a variety of forms. Therefore, while the embodiments have been described in connection with particular examples thereof, the true scope of the embodiments should not be so limited since other modifications will become apparent to the skilled practitioner upon a study of the drawings, specification, and following claims.
