Patent Application
20240020103
Benefit is claimed under 35 U.S.C. 119(a)-(d) to Foreign Application Serial No. 202241040257 filed in India entitled “PARALLELIZING DATA PROCESSING UNIT PROVISIONING”, on Jul. 13, 2022, by VMware, Inc., which is herein incorporated in its entirety by reference for all purposes.
Modern computing devices often have dedicated offload cards installed in order to improve the performance or throughput for various tasks. These offload cards can be quite sophisticated, with their own, processors, memory, and operating system. The installation of an operating system or firmware on the offload cards is often done when the operating system on the host machine is also installed. For example, an installer process on the host machine can provision the offload cards as a part of an installation flow where configuration of the host machine is completed and where other hardware and software components on the host machine are configured or installed. Accordingly, if there are multiple offload cards within or accessible to the host machine that require configuration or provisioning, the process of provisioning these offload cards can create a bottleneck that slows the provisioning of the host machine for use by users or workloads. This can unacceptably slow or delay the availability of the host machine to process workloads on behalf of an enterprise.
Many aspects of the present disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, with emphasis instead being placed upon clearly illustrating the principles of the disclosure. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.
Disclosed are various approaches for coordinating the installation of an operating system onto a host machine as well as a respective operating system installed onto data processing units (DPU) of an operating system installed on a host machine. A DPU can be an offload card or a smart network interface card installed on a host machine that has its own CPU and other resources that require provisioning in addition to the host machine. During installation of an operating system on a host machine, the installation workflow can also require installation of an additional operating system or other configuration of a DPU installed in a host machine. In some cases, there can be many DPU's installed in a host machine that require configuration or provisioning. Accordingly, provisioning these DPU's can allow the overall provisioning of a host machine in which the DPU's are installed.
To resolve these issues, the various embodiments of the present disclosure cause the installation flow that installs an operating system on the host machine and the operating system installed on the offload cards to be completed in parallel. By parallelizing these operations, provisioning time of the host machine can be drastically reduced, thereby speeding the provisioning process for these host machines. In one example, the installation flow can install a bare metal hypervisor on the host machine and the same or a different operating system on the DPU's installed in the host machine.
In the following discussion, a general description of the system and its components is provided, followed by a discussion of the operation of the same. Although the following discussion provides illustrative examples of the operation of various components of the present disclosure, the use of the following illustrative examples does not exclude other implementations that are consistent with the principals disclosed by the following illustrative examples.
The host operating system 113 can include any system software that manages the operation of computer hardware and software resources of the host machine 103. The host operating system 113 can also provide various services or functions to computer programs that are executed by the host machine 103. For example, the host operating system 113 may schedule the operation of tasks or processes by the processor of the host machine 103. The host operating system 113 may also provide virtual memory management functions to allow processes executing on the host machine 103 to have its own logical or virtual address space, which the host operating system 113 can map to physical addresses in the memory of the host machine 103. When referring to the host operating system 113, the host operating system 113 can include both hypervisors and/or any other system software that manages computer hardware and software resources.
The host boot loader 116 can represent a program responsible for booting the host operating system 113 in response to the host machine 103 being powered on. Once execution of the host boot loader 116 is initiated, the bootloader can select the host boot image 123 to boot the host operating system 113. In some examples, the host bootloader 116 can select an alternative host boot image to select in the event that the host boot image 123 is inoperative or defective. The host bootloader 116 can make such a determination by detecting that the operating system of the host machine 103 fails to return a success signal upon bootup.
The host boot image 123 represents a disk image containing a copy of the current version of the host operating system 113 to be executed by the host machine 103. The host boot image 123 can also include configuration information and state information, such as whether the most recent boot using the host boot image 123 had failed.
Examples of the disclosure can allow an installation application or service to install a fresh operating system or an updated operating system onto the host machine 103. can represent a disk image containing a previous version of the host operating system 113 to be executed by the host machine 103. A user can initiate provisioning of the host machine 103 to install software on the device, such as a bare-metal hypervisor that allows the host machine 103 to execute virtual machines that can support workloads such as virtual desktop infrastructure, server infrastructure, datacenter operations, or any other workloads needed by a customer provisioning the host machine 103. The host machine 103 can represent a server that is being provisioned for an enterprise.
The host operating system 113 can execute an installer process that can orchestrate the installation process. The process is referred to herein as the orchestrator 128. The orchestrator 128 can oversee installation of a host boot image 123 on the host machine 103. The orchestrator 128 can also oversee provisioning of one or more DPU 106 of the host machine 103.
The host firmware 119 can include software embedded in the host machine 103 to provide a standardized operating environment for more complex software executing on the host machine 103. For example, the PC-compatible Basic Input/Output System (PC-BIOS) used by many desktops, laptops, and servers initializes and tests system hardware components, enables or disables hardware functions as specified in the PC-BIOS configuration, and the loads the host bootloader 116 from memory to initialize the host operating system 113 of the host machine 103. The PC-BIOS also provides a hardware abstraction layer (HAL) for keyboard, display, and other input/output devices which may be used by the host operating system 113 of the host machine 103. The Unified Extensible Firmware Interface (UEFI) provides similar functions as the BIOS, as well as various additional functions such as Secure Boot, a shell environment for interacting with the host machine 103, network connectivity for the host machine 103, and various other functions.
The DPU 106 can represent an offload card installed on the host machine 103 to accelerate the processing of various types of compute workloads. Accordingly, the DPU 106 can include at least one processor, memory, and (in some implementations), one or more network interfaces. DPUs 106 can be used, for example, to accelerate network packet processing (e.g., for a firewall, software defined switch, etc.), input/output operations for local or network storage, or other computational workloads. In other instances, the DPU 106 can be used to execute applications that would typically be executed by the central processor unit (CPU) of the host machine 103, to make the resources of the CPU of the host machine 103 available for other tasks. For example, the DPU 106 could execute a hypervisor so that the resources of the CPU of the host machine 103 could be fully dedicated to the guests executing on the host machine 103. Accordingly, in various embodiments, the DPU 106 could execute a DPU operating system 129, a DPU firmware 133, and a DPU bootloader 136.
The DPU operating system 129 can include any system software that manages the operation of computer hardware and software resources of the DPU 106. The DPU operating system 129 can also provide various services or functions to computer programs that are executed by the DPU 106. For example, the DPU operating system 129 may schedule the operation of tasks or processes by the processor of the DPU 106. This could include network packet processing, network packet processing (e.g., for a firewall, software defined switch, etc.), input/output operations for local or network storage, or other computational workloads.
In implementations where the functionality of a hypervisor is implemented by the DPU 106, the DPU operating system 129 may also provide virtual memory management functions to allow processes executing on the host machine 103 to have its own logical or virtual address space, which the DPU operating system 129 can map to physical addresses in the memory of the host machine 103. When referring to the DPU operating system 129, the DPU operating system 129 can include both hypervisors and/or any other system software that manages computer hardware and software resources.
The DPU firmware 133 can include software embedded in the DPU 106 to provide a standardized operating environment for more complex software executing on the DPU 106. For example, the PC-compatible Basic Input/Output System (PC-BIOS) used by many desktops, laptops, and servers initializes and tests system hardware components, enables or disables hardware functions as specified in the PC-BIOS configuration, and the loads the DPU bootloader 136 from memory to initialize the DPU operating system 129 of the DPU 106. The PC-BIOS also provides a hardware abstraction layer (HAL) for keyboard, display, and other input/output devices which may be used by the DPU operating system 129 of the DPU 106. The Unified Extensible Firmware Interface (UEFI) provides similar functions as the BIOS, as well as various additional functions such as Secure Boot, a shell environment for interacting with the DPU 106, network connectivity for the DPU 106, and various other functions.
The DPU bootloader 136 can represent a program responsible for booting the DPU operating system 129 in response to the DPU 106 being powered on. Once execution of the DPU bootloader 136 is initiated, the bootloader can select either the DPU boot image 139 or a DPU alternate boot image to boot the DPU operating system 129.
The DPU boot image 139 represents a disk image containing a copy of the current version of the DPU operating system 129 to be executed by the DPU 106. The DPU boot image 139 can also include configuration information and state information, such as whether the most recent boot using the DPU boot image 139 had failed.
The orchestrator 128 can manage the installation process of a DPU boot image 139 on a DPU 106. In one example, the orchestrator 128 can create or provide an installation executable or image that can be installed by the DPU bootloader 136 or another process on the DPU 106.
The orchestrator 128 can execute a server process from which the DPU 106 and/or BMC 109 can retrieve an installation image and install the DPU operating system 129 onto the DPU 106 when the host machine 103 is being provisioned. In examples of this disclosure, the process of spawning a server process to provide to the respective DPU's 106 in the host machine 103 can be executed or continued in parallel with an installation flow that install and/or configures the host operating system 113 on the host machine 103. Additionally, in the case of multiple DPU's 106 on the host machine 103, the respective server processes can be executed in parallel with one another. In this way, provisioning each of the respective DPU's 106 in the host machine 103 should not act as a bottleneck that slows the installation and configuration process of the host machine 103. The server process can represent an HTTP server, an FTP server, or any other server that supports file transfer between network nodes.
The BMC 109 represents a specialized microcontroller embedded on the motherboard of the host machine 103 that provides an interface between system management software (such as the host operating system 113 or host firmware 119) and the hardware of the host machine 103. This can include, for example, providing a serial console over a network connection or other out of band communications and control mechanisms for the host machine 103. The BMC 109 can also provide out of band communications channels between hardware components of the host machine 103, such as between the DPU 106 and other components of the host machine 103. In some implementations, the BMC 109 can include its own memory, processor, and optimized embedded firmware.
The orchestrator 128 represents a process or application that can facilitate installation of software on the host machine 103. The orchestrator 128 can be a module within an installer application that can install or configure the host operating system 113 on the host machine 103. The orchestrator 128 can also provide an installation image or application that a DPU 106 can utilize to install or provision the DPU operating system 129 on the DPU 106.
Referring next to
Beginning with block 203, the host operating system 113 can spawn a thread for the orchestrator 128. The host operating system 113, at this stage, can be an application or process that is executing from a network or an external drive, such as an operating system installer. The installer can implement an installation workflow that installs a new operating system on the host machine 103, such as a bare metal hypervisor that can provide virtual machine capabilities to the host machine 103.
At step 206, the orchestrator 128 can generate a DPU operating system 129 installation image. The DPU operating system 129 installation image can be provided to a respective DPU 106 in the host machine 103 so that the DPU 106 can be provisioned with an operating system, such as a bare metal hypervisor or a complementary operating system to a bare metal hypervisor running on the host machine 103. The DPU operating system 129 installation image can also be obtained from an installation image that is utilized to install a host machine 103 operating system. The DPU operating system 129 installation image can also be obtained from a network source that is remotely located from the host machine 103.
At step 208, the host operating system 113 or the orchestrator 128 can continue execution of a host machine 103 installation flow that installs a host operating system 113 on the host machine 103 or that configures and/or provisions the host operating system 113 on the host machine 103.
At step 209, the orchestrator 128 can initiate a server process to host the DPU operating system 129 installation image generated or obtained at step 206. The server process can be running on the host machine 103, and the DPU 106 can communicate with the server process using a network stack that is available to the DPU 106. The BMC 109 can provide the ability for the DPU 106 and the host machine 103 to communicate using a network stack.
In one implementation, the orchestrator 128 can create a server process for each DPU 106 in the host machine. In another implementation, the orchestrator 128 can create a single server process that can handle requests from multiple DPU 106.
Accordingly, at step 211, the orchestrator 128 can provide the uniform resource locator (URL) or network address of the server process to the BMC 109. The BMC 109 can provide a networking stack or networking capability to the DPU 106 so that the host machine 103 and the DPU 106 can communicate using networking protocols.
At step 215, the DPU 106 can download the DPU operating system 129 installation image provided by the server process created by the orchestrator 128. The DPU operating system 129 installation image can represent an installation image that can be installed by the DPU bootloader 136 or another provisioning service on the DPU 106, such as a process provided by the DPU firmware 133 to install an operating system on the DPU 106. The DPU operating system 129 installation image can represent an ISO image or an executable file in a format that is compatible with the DPU firmware 133 or the DPU bootloader 136 according to the particular specifications of the respective DPU 106.
At step 217, the DPU 106 can initiate a DPU installation flow. The DPU installation flow can represent an installer that installs and configures an DPU operating system 129 onto the DPU 106. The DPU operating system 129 can execute the installer workflow so that the installer can install a bare metal hypervisor, a server operating system, a network stack, or any other software component or operating system onto the DPU 106 so that the DPU 106 can work with the host machine 103 to facilitate user workloads and other tasks. The DPU installer workflow can install a DPU boot image 139 onto the DPU 106 that the DPU bootloader 136 can boot whenever DPU 106 is powered up or rebooted.
At step 219, the DPU installer workflow can provide an indication of completion to the BMC 109. For example, the, the DPU bootloader 136 can boot a DPU boot image when the installer workflow has completed so that the DPU 106 is powered on and begins to boot. The DPU operating system 129 can provide a success signal upon bootup of the DPU 106 if the DPU 106 successfully boots the DPU boot image 139.
However, if the DPU operating system 129 fails to successfully boot from the DPU boot image 139, then the DPU 106 may not provide an indication of completion to the BMC 109 at step 219. For example, the BMC 109 or orchestrator 128 can determine after a timeout period that the installer did not successfully complete. In this scenario, the BMC 109 or the orchestrator 128 can determine that the DPU installer workflow was unsuccessful and take one or more remedial actions. In one example, the orchestrator 128 can report the failure of the DPU installation workflow to the host operating system 113 or a user monitoring the installation flow implemented by the orchestrator 128 so that the user can intervene. In another scenario, the orchestrator 128 can restart the DPU installation workflow on the DPU 106 or power cycling the DPU 106.
The host bootloader 116 can determine whether the DPU operating system 129 has successfully booted by polling the BMC 109 to determine whether the DPU operating system 129 has sent a ready signal to the BMC 109. Failure to receive a ready signal from the DPU operating system 129 within a predefined time period could serve as an indicator that the DPU operating system 129 has failed to boot.
Next, at block 222, the BMC 109 can provide the indication of completion of the DPU installation flow to the orchestrator 128. As noted above, the orchestrator 128 can monitor potentially multiple DPU installation flows corresponding to multiple DPU 106 in the host machine 103. The steps shown in steps 213, 215, 217, 219 and/or 222 can be performed in parallel with an installation flow carried out by the orchestrator 128 or another process to install and configure a host operating system 113 on the host machine 103. In this way, the DPU installation flow for potentially multiple DPU 106 and an installation flow for the host operating system 113 can operate in parallel, which can speed the provisioning of the host machine 103 relative to conducting the respective installation flows in series.
Continuing the example of
At step 231, the host machine 103 provisioning and configuration can be completed. In one example, the orchestrator 128 can determine that the installation flow for the host operating system 113 has completed and that the DPU installation flow for the respective DPU 106 in the host machine 103 are also completed.
At step 233, the host machine 103 can reboot upon completion of host machine 103 provisioning and configuration. At step 235, the DPU 106 can reboot upon completion of host machine 103 provisioning and configuration. In one example, reboot of the host machine 103 and the DPU 106 can be performed in parallel. Additionally, in some implementations of a host machine 103, there can be multiple DPU 106 installed in a host machine 103.
Several software components previously discussed are stored in the memory of the respective computing devices and are executable by the processor of the respective computing devices. In this respect, the term “executable” means a program file that is in a form that can ultimately be run by the processor. Examples of executable programs can be a compiled program that can be translated into machine code in a format that can be loaded into a random access portion of the memory and run by the processor, source code that can be expressed in proper format such as object code that is capable of being loaded into a random access portion of the memory and executed by the processor, or source code that can be interpreted by another executable program to generate instructions in a random access portion of the memory to be executed by the processor. An executable program can be stored in any portion or component of the memory, including random access memory (RAM), read-only memory (ROM), hard drive, solid-state drive, Universal Serial Bus (USB) flash drive, memory card, optical disc such as compact disc (CD) or digital versatile disc (DVD), floppy disk, magnetic tape, or other memory components.
The memory includes both volatile and nonvolatile memory and data storage components. Volatile components are those that do not retain data values upon loss of power. Nonvolatile components are those that retain data upon a loss of power. Thus, the memory can include random access memory (RAM), read-only memory (ROM), hard disk drives, solid-state drives, USB flash drives, memory cards accessed via a memory card reader, floppy disks accessed via an associated floppy disk drive, optical discs accessed via an optical disc drive, magnetic tapes accessed via an appropriate tape drive, or other memory components, or a combination of any two or more of these memory components. In addition, the RAM can include static random access memory (SRAM), dynamic random access memory (DRAM), or magnetic random access memory (MRAM) and other such devices. The ROM can include a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or other like memory device.
Although the applications and systems described herein can be embodied in software or code executed by general purpose hardware as discussed above, as an alternative the same can also be embodied in dedicated hardware or a combination of software/general purpose hardware and dedicated hardware. If embodied in dedicated hardware, each can be implemented as a circuit or state machine that employs any one of or a combination of a number of technologies. These technologies can include, but are not limited to, discrete logic circuits having logic gates for implementing various logic functions upon an application of one or more data signals, application specific integrated circuits (ASICs) having appropriate logic gates, field-programmable gate arrays (FPGAs), or other components, etc. Such technologies are generally well known by those skilled in the art and, consequently, are not described in detail herein.
The flowcharts and sequence diagrams show the functionality and operation of an implementation of portions of the various embodiments of the present disclosure. If embodied in software, each block can represent a module, segment, or portion of code that includes program instructions to implement the specified logical function(s). The program instructions can be embodied in the form of source code that includes human-readable statements written in a programming language or machine code that includes numerical instructions recognizable by a suitable execution system such as a processor in a computer system. The machine code can be converted from the source code through various processes. For example, the machine code can be generated from the source code with a compiler prior to execution of the corresponding application. As another example, the machine code can be generated from the source code concurrently with execution with an interpreter. Other approaches can also be used. If embodied in hardware, each block can represent a circuit or a number of interconnected circuits to implement the specified logical function or functions.
Although the flowcharts and sequence diagrams show a specific order of execution, it is understood that the order of execution can differ from that which is depicted. For example, the order of execution of two or more blocks can be scrambled relative to the order shown. Also, two or more blocks shown in succession can be executed concurrently or with partial concurrence. Further, in some embodiments, one or more of the blocks shown in the flowcharts and sequence diagrams can be skipped or omitted. In addition, any number of counters, state variables, warning semaphores, or messages might be added to the logical flow described herein, for purposes of enhanced utility, accounting, performance measurement, or providing troubleshooting aids, etc. It is understood that all such variations are within the scope of the present disclosure.
Also, any logic or application described herein that includes software or code can be embodied in any non-transitory computer-readable medium for use by or in connection with an instruction execution system such as a processor in a computer system or other system. In this sense, the logic can include statements including instructions and declarations that can be fetched from the computer-readable medium and executed by the instruction execution system. In the context of the present disclosure, a “computer-readable medium” can be any medium that can contain, store, or maintain the logic or application described herein for use by or in connection with the instruction execution system. Moreover, a collection of distributed computer-readable media located across a plurality of computing devices (e.g, storage area networks or distributed or clustered filesystems or databases) may also be collectively considered as a single non-transitory computer-readable medium.
The computer-readable medium can include any one of many physical media such as magnetic, optical, or semiconductor media. More specific examples of a suitable computer-readable medium would include, but are not limited to, magnetic tapes, magnetic floppy diskettes, magnetic hard drives, memory cards, solid-state drives, USB flash drives, or optical discs. Also, the computer-readable medium can be a random access memory (RAM) including static random access memory (SRAM) and dynamic random access memory (DRAM), or magnetic random access memory (MRAM). In addition, the computer-readable medium can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or other type of memory device.
Further, any logic or application described herein can be implemented and structured in a variety of ways. For example, one or more applications described can be implemented as modules or components of a single application. Further, one or more applications described herein can be executed in shared or separate computing devices or a combination thereof. For example, a plurality of the applications described herein can execute in the same computing device, or in multiple computing devices in the same computing environment.
Disjunctive language such as the phrase “at least one of X, Y, or Z,” unless specifically stated otherwise, is otherwise understood with the context as used in general to present that an item, term, etc., can be either X, Y, or Z, or any combination thereof (e.g., X; Y; Z; X or Y; X or Z; Y or Z; X, Y, or Z; etc.). Thus, such disjunctive language is not generally intended to, and should not, imply that certain embodiments require at least one of X, at least one of Y, or at least one of Z to each be present.
It should be emphasized that the above-described embodiments of the present disclosure are merely possible examples of implementations set forth for a clear understanding of the principles of the disclosure. Many variations and modifications can be made to the above-described embodiments without departing substantially from the spirit and principles of the disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202241040257 | Jul 2022 | IN | national |
