Patent Application
20250021366
The disclosure relates generally to virtual machine memory management.
As the value and use of information continues to increase, individuals and businesses seek additional ways to process and store information. One option available to users is information handling systems. An information handling system generally processes, compiles, stores, and/or communicates information or data for business, personal, or other purposes thereby allowing users to take advantage of the value of the information. Because technology and information handling needs and requirements vary between different users or applications, information handling systems may also vary regarding what information is handled, how the information is handled, how much information is processed, stored, or communicated, and how quickly and efficiently the information may be processed, stored, or communicated. The variations in information handling systems allow for information handling systems to be general or configured for a specific user or specific use such as financial transaction processing, airline reservations, enterprise data storage, or global communications. In addition, information handling systems may include a variety of hardware and software components that may be configured to process, store, and communicate information and may include one or more computer systems, data storage systems, and networking systems.
Innovative aspects of the subject matter described in this specification may be embodied in a method of virtual machine memory management, the method including assigning, for each virtual machine of a plurality of virtual machines operating at a compute node, one or more respective first memory modules of a plurality of first memory modules to the virtual machine, the plurality of first memory modules operating at a storage node; accessing, by the plurality of virtual machines and over a first communication channel between the compute node and the storage node, the respective first memory modules; detecting, for a particular virtual machine of the plurality of virtual machines, a loss of connectivity of the particular virtual machine with respective first memory modules assigned to the particular virtual machine; in response to detecting the loss of connectivity of the particular virtual machine with respective first memory modules assigned to the particular virtual machine: assigning, for the particular virtual machine, one or more second memory modules of a plurality of second memory modules to the particular virtual machine, the plurality of second memory modules operating at the compute node; establishing a second communication channel between a baseband management controller (BMC) of the storage node and a remote access controller (RAC) of the computing node; and transferring, over the second communication channel, data stored at the respective first memory modules assigned to the particular virtual machine to the second memory modules assigned to the particular virtual machine.
Other embodiments of these aspects include corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices.
These and other embodiments may each optionally include one or more of the following features. For instance, after transferring the data stored at the respective first memory modules assigned to the particular virtual machine to the assigned second memory modules, restarting the particular virtual machine. Maintaining operation of the particular virtual machine after restarting. Assigning, for the particular virtual machine, one or more second memory modules of a plurality of second memory modules to the particular virtual machine further includes: identifying non-volatile memory modules of the one or more second memory modules; identifying volatile memory modules of the one or more second memory modules; determining that one or more of the non-volatile memory modules of the one or more second memory modules is available; in response to determining that one or more of the non-volatile memory modules of the one or more second memory modules is available, assigning, for the particular virtual machine, one or more of the non-volatile memory modules of the plurality of second memory modules to the particular virtual machine; and transferring, over the second communication channel, data stored at the respective first memory modules assigned to the particular virtual machine to the non-volatile memory modules assigned to the particular virtual machine. Determining that the non-volatile memory modules of the one or more second memory modules are unavailable; in response to determining that the non-volatile memory modules of the one or more second memory modules are unavailable, assigning, for the particular virtual machine, one or more of the volatile memory modules of the plurality of second memory modules to the particular virtual machine; and transferring, over the second communication channel, data stored at the respective first memory modules assigned to the particular virtual machine to the non-volatile memory modules assigned to the particular virtual machine. Determining that the non-volatile memory modules and the non-volatile memory modules of the one or more second memory modules are unavailable; and in response to determining that the non-volatile memory modules and the non-volatile memory modules of the one or more second memory modules are unavailable, assigning, for the particular virtual machine, one or more additional memory modules of a plurality of additional memory modules to the particular virtual machine, the plurality of additional memory modules operating at an additional storage node. The first memory modules are non-volatile dual inline memory modules (NVDIMM) over fabric.
The details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.
This disclosure discusses methods and systems for virtual machine memory management. Specifically, this disclosure discusses a system and a method including assigning, for each virtual machine of a plurality of virtual machines operating at a compute node, one or more respective first memory modules of a plurality of first memory modules to the virtual machine, the plurality of first memory modules operating at a storage node; accessing, by the plurality of virtual machines and over a first communication channel between the compute node and the storage node, the respective first memory modules; detecting, for a particular virtual machine of the plurality of virtual machines, a loss of connectivity of the particular virtual machine with respective first memory modules assigned to the particular virtual machine; in response to detecting the loss of connectivity of the particular virtual machine with respective first memory modules assigned to the particular virtual machine: assigning, for the particular virtual machine, one or more second memory modules of a plurality of second memory modules to the particular virtual machine, the plurality of second memory modules operating at the compute node; establishing a second communication channel between a baseband management controller (BMC) of the storage node and a remote access controller (RAC) of the computing node; and transferring, over the second communication channel, data stored at the respective first memory modules assigned to the particular virtual machine to the second memory modules assigned to the particular virtual machine.
In the following description, details are set forth by way of example to facilitate discussion of the disclosed subject matter. It should be apparent to a person of ordinary skill in the field, however, that the disclosed embodiments are examples and not exhaustive of all possible embodiments.
As used herein, a reference numeral refers to a class or type of entity, and any letter following such reference numeral refers to a specific instance of a particular entity of that class or type. Thus, for example, a hypothetical entity referenced by ‘12A’ may refer to a particular instance of a particular class/type, and the reference ‘12’ may refer to a collection of instances belonging to that particular class/type or any one instance of that class/type in general.
An information handling system (IHS) may include a hardware resource or an aggregate of hardware resources operable to compute, classify, process, transmit, receive, retrieve, originate, switch, store, display, manifest, detect, record, reproduce, handle, and/or utilize various forms of information, intelligence, or data for business, scientific, control, entertainment, or other purposes, according to one or more embodiments. For example, an IHS may be a personal computer, a desktop computer system, a laptop computer system, a server computer system, a mobile device, a tablet computing device, a personal digital assistant (PDA), a consumer electronic device, an electronic music player, an electronic camera, an electronic video player, a wireless access point, a network storage device, or another suitable device and may vary in size, shape, performance, functionality, and price. In one or more embodiments, a portable IHS may include or have a form factor of that of or similar to one or more of a laptop, a notebook, a telephone, a tablet, and a PDA, among others. For example, a portable IHS may be readily carried and/or transported by a user (e.g., a person). In one or more embodiments, components of an IHS may include one or more storage devices, one or more communications ports for communicating with external devices as well as various input and output (I/O) devices, such as a keyboard, a mouse, and a video display, among others. In one or more embodiments, IHS may include one or more buses operable to transmit communication between or among two or more hardware components. In one example, a bus of an IHS may include one or more of a memory bus, a peripheral bus, and a local bus, among others. In another example, a bus of an IHS may include one or more of a Micro Channel Architecture (MCA) bus, an Industry Standard Architecture (ISA) bus, an Enhanced ISA (EISA) bus, a Peripheral Component Interconnect (PCI) bus, HyperTransport (HT) bus, an inter-integrated circuit (I2C) bus, a serial peripheral interface (SPI) bus, a low pin count (LPC) bus, an enhanced serial peripheral interface (eSPI) bus, a universal serial bus (USB), a system management bus (SMBus), and a Video Electronics Standards Association (VESA) local bus, among others.
In one or more embodiments, an IHS may include firmware that controls and/or communicates with one or more hard drives, network circuitry, one or more memory devices, one or more I/O devices, and/or one or more other peripheral devices. For example, firmware may include software embedded in an IHS component utilized to perform tasks. In one or more embodiments, firmware may be stored in non-volatile memory, such as storage that does not lose stored data upon loss of power. In one example, firmware associated with an IHS component may be stored in non-volatile memory that is accessible to one or more IHS components. In another example, firmware associated with an IHS component may be stored in non-volatile memory that may be dedicated to and includes part of that component. For instance, an embedded controller may include firmware that may be stored via non-volatile memory that may be dedicated to and includes part of the embedded controller.
An IHS may include a processor, a volatile memory medium, non-volatile memory media, an I/O subsystem, and a network interface. Volatile memory medium, non-volatile memory media, I/O subsystem, and network interface may be communicatively coupled to processor. In one or more embodiments, one or more of volatile memory medium, non-volatile memory media, I/O subsystem, and network interface may be communicatively coupled to processor via one or more buses, one or more switches, and/or one or more root complexes, among others. In one example, one or more of a volatile memory medium, non-volatile memory media, an I/O subsystem, a and network interface may be communicatively coupled to the processor via one or more PCI-Express (PCIe) root complexes. In another example, one or more of an I/O subsystem and a network interface may be communicatively coupled to processor via one or more PCIe switches.
In one or more embodiments, the term “memory medium” may mean a “storage device”, a “memory”, a “memory device”, a “tangible computer readable storage medium”, and/or a “computer-readable medium”. For example, computer-readable media may include, without limitation, storage media such as a direct access storage device (e.g., a hard disk drive, a floppy disk, etc.), a sequential access storage device (e.g., a tape disk drive), a compact disk (CD), a CD-ROM, a digital versatile disc (DVD), a random access memory (RAM), a read-only memory (ROM), a one-time programmable (OTP) memory, an electrically erasable programmable read-only memory (EEPROM), and/or a flash memory, a solid state drive (SSD), or any combination of the foregoing, among others.
In one or more embodiments, one or more protocols may be utilized in transferring data to and/or from a memory medium. For example, the one or more protocols may include one or more of small computer system interface (SCSI), Serial Attached SCSI (SAS) or another transport that operates with the SCSI protocol, advanced technology attachment (ATA), serial ATA (SATA), a USB interface, an Institute of Electrical and Electronics Engineers (IEEE) 1394 interface, a Thunderbolt interface, an advanced technology attachment packet interface (ATAPI), serial storage architecture (SSA), integrated drive electronics (IDE), or any combination thereof, among others.
A volatile memory medium may include volatile storage such as, for example, RAM, DRAM (dynamic RAM), EDO RAM (extended data out RAM), SRAM (static RAM), etc. One or more of non-volatile memory media may include nonvolatile storage such as, for example, a read only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM, NVRAM (non-volatile RAM), ferroelectric RAM (FRAM), a magnetic medium (e.g., a hard drive, a floppy disk, a magnetic tape, etc.), optical storage (e.g., a CD, a DVD, a BLU-RAY disc, etc.), flash memory, a SSD, etc. In one or more embodiments, a memory medium can include one or more volatile storages and/or one or more nonvolatile storages.
In one or more embodiments, a network interface may be utilized in communicating with one or more networks and/or one or more other information handling systems. In one example, network interface may enable an IHS to communicate via a network utilizing a suitable transmission protocol and/or standard. In a second example, a network interface may be coupled to a wired network. In a third example, a network interface may be coupled to an optical network. In another example, a network interface may be coupled to a wireless network. In one instance, the wireless network may include a cellular telephone network. In a second instance, the wireless network may include a satellite telephone network. In another instance, the wireless network may include a wireless Ethernet network (e.g., a Wi-Fi network, an IEEE 802.11 network, etc.).
In one or more embodiments, a network interface may be communicatively coupled via a network to a network storage resource. For example, the network may be implemented as, or may be a part of, a storage area network (SAN), personal area network (PAN), local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wireless local area network (WLAN), a virtual private network (VPN), an intranet, an Internet or another appropriate architecture or system that facilitates the communication of signals, data and/or messages (generally referred to as data). For instance, the network may transmit data utilizing a desired storage and/or communication protocol, including one or more of Fibre Channel, Frame Relay, Asynchronous Transfer Mode (ATM), Internet protocol (IP), other packet-based protocol, Internet SCSI (iSCSI), or any combination thereof, among others.
In one or more embodiments, a processor may execute processor instructions in implementing at least a portion of one or more systems, at least a portion of one or more flowcharts, at least a portion of one or more methods, and/or at least a portion of one or more processes. In one example, a processor may execute processor instructions from one or more memory media in implementing at least a portion of one or more systems, at least a portion of one or more flowcharts, at least a portion of one or more methods, and/or at least a portion of one or more processes. In another example, a processor may execute processor instructions via a network interface in implementing at least a portion of one or more systems, at least a portion of one or more flowcharts, at least a portion of one or more methods, and/or at least a portion of one or more processes.
In one or more embodiments, a processor may include one or more of a system, a device, and an apparatus operable to interpret and/or execute program instructions and/or process data, among others, and may include one or more of a microprocessor, a microcontroller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), and another digital or analog circuitry configured to interpret and/or execute program instructions and/or process data, among others. In one example, a processor may interpret and/or execute program instructions and/or process data stored locally (e.g., via memory media and/or another component of an IHS). In another example, a processor may interpret and/or execute program instructions and/or process data stored remotely (e.g., via a network storage resource).
In one or more embodiments, an I/O subsystem may represent a variety of communication interfaces, graphics interfaces, video interfaces, user input interfaces, and/or peripheral interfaces, among others. For example, an I/O subsystem may include one or more of a touch panel and a display adapter, among others. For instance, a touch panel may include circuitry that enables touch functionality in conjunction with a display that is driven by a display adapter.
A non-volatile memory medium may include an operating system (OS) and applications (APPs). In one or more embodiments, one or more of an OS and APPs may include processor instructions executable by a processor. In one example, a processor may execute processor instructions of one or more of OS and APPs via a non-volatile memory medium. In another example, one or more portions of the processor instructions of one or more of an OS and APPs may be transferred to a volatile memory medium and a processor may execute the one or more portions of the processor instructions.
Non-volatile memory medium may include information handling system firmware (IHSFW). In one or more embodiments, IHSFW may include processor instructions executable by a processor. For example, IHSFW may include one or more structures and/or one or more functionalities of and/or compliant with one or more of a basic input/output system (BIOS), an Extensible Firmware Interface (EFI), a Unified Extensible Firmware Interface (UEFI), and an Advanced Configuration and Power Interface (ACPI), among others. In one instance, a processor may execute processor instructions of IHSFW via non-volatile memory medium. In another instance, one or more portions of the processor instructions of IHSFW may be transferred to volatile memory medium, and processor may execute the one or more portions of the processor instructions of IHSFW via volatile memory medium.
Data centers may have large numbers of information handling systems such as servers for processing information. A data center facility may have one or more floors with each floor having racks of servers. A server may be processing a set of information independently or a group of servers may be working on the same set of information.
Turning now to
Turning to
Firmware in RAC 14 may include a set of built in features 30 executable by a processor in RAC 14. For example, each server 12 has a power supply unit (PSU) for receiving power from a power source and transforming the power into usable power by the server 12. Firmware in RAC 14 includes power control 30-1 as a feature 30 for monitoring of power received by the PSU to determine when server 12 has switched power modes and allow for remote switching of server 12 between power modes. Similarly, server 12 has a fan for cooling server 12 and firmware in RAC 14 includes thermal control 30-2 as a feature 30 for monitoring of temperatures in server 12 to determine if the fan is operating, determining when to operate the fan and enabling remote operation of the fan.
Firmware in RAC 14 may also store hardware abstraction layer (HAL) 22 for communication with and monitoring of peripheral devices. Within HAL 22, features such as Inter-Integrated Circuit (I2C) protocol 22-1, Improved Inter-Integrated Circuit (I3C) protocol 22-2, Serial Peripheral Interface (SPI) 22-3, Reduced Media-Independent Interface (RMII) 22-4, Peripheral Connect Interface Express Vendor Defined Message (PCIe VDM) 22-5 and management component transport protocol (MCTP) 22-6 allow communications with chips, processors and microcontrollers in server 12.
Server 12 includes hardware 24 such as processors for processing information, memory for storing information, a fan for cooling server 12 including devices and components in server 12, a network integrated circuit (NIC) for communications, other controllers such as RAID controllers, and Complex Logic Programmable Devices (CPLDs). Accordingly, Firmware in RAC 14 may include other features such as Basic Input Output Service (BIOS) 26-1, NIC 26-2, fan control 26-3, PSU control 26-4 for operating a PSU, RAID feature 26-5 for managing a RAID and CPLD feature 26-6 for monitoring and managing a CPLD.
Firmware in RAC 14 may typically include other features 30 for monitoring and managing servers 12 in data centers 10. The examples depicted in
Web Graphical User Interface (GUI) 30-1 is a web-based application that processes network events for displaying the data in a graphical format for users to view on an attached display. Redfish® 30-2 is an application programming interface (API) that uses RESTful semantics to access data defined in model format for systems management. Remote Access Controller admin (RACADM) feature 30-3 is a scriptable interface to allow remote configuration of RAC 14. Simple Network Management Protocol (SNMP) feature 30-4 may be used to collect data related to network changes and determine the status of network-connected devices. Hardware inventory feature 30-5 may maintain an inventory and properties of all hardware installed on server 12. Software inventory feature 30-6 may maintain an inventory and versions of all software running on server 12. Firmware update feature 30-7 may maintain a list of firmware including versions and facilitate updates of any firmware on server 12. Monitoring feature 30-8 may monitor operation of components or devices in server 12 and record values or may define what operations are to be monitored and how the monitoring should occur. RAC 14 may include database 32 for storing information about components or devices in server 12. Network Controller-Sideband Interface (e.g., NC-SI 1.2) feature 30-10 defines a control communication protocol between a baseboard management controller (BMC) and one or more Network Interface Controllers (NICs). Platform Level Data Model (PLDM) feature 30-11 defines the contents of a firmware update package. Security Protocol and Data Models (SPDM) feature 30-12 enables authentication, attestation and key exchange for enabling security of server 12. Management Control Transport Protocol (MCTP) feature 30-13 stores message formats, transport descriptions, message exchange patterns and endpoint characteristics related to communications between components.
Servers 12 described above have several shortcomings. All features 30 are stored in memory in each RAC 14 of each server 12, regardless of whether a feature 30 will be used for a particular server 12. Furthermore, features 30 are getting more robust and require more memory to manage devices in servers 12. For example, RAC 14 in some generations of servers 12 may have 512 Megabytes (MB) of memory to store all features 30, RAC 14 in later generations of servers 12 may have 1 Gigabyte (GB) of memory to store all features 30, and RAC 14 in later generations of servers 12 may have 2 GB of memory to store all features 30. Although each server 12 is provided with the full suite of features 30, many customers or servers 12 require a limited set of features 30. For security reasons, some users prefer to have only firmware code which they know they will use. Subscription-based licensing, in which a user pays for only the features needed, may be preferred. However, only about 35% of datacenter servers 12 contain the latest firmware for RAC 14, and features 30 are often unnecessarily tied to specific hardware capabilities. Customization of features 30 via Software Developer Kits (SDK) may be limited or unavailable due to hardware capabilities or software requirements. Some devices (e.g., Baseboard Management Controllers (BMCs)) are expected to be commoditized, resulting in greater complexity and/or more features 30 that would need to be installed on each RAC 14.
Turning to
Referring to
Data center 310-2 may have a second plurality of servers 312 (e.g., 2 to N) divided into Q groups 340, wherein group 340-4 may have a first set of servers 312 (e.g., 1 to D), group 340-2 may have a second set of servers 312 (e.g., 1 to E) and group 340-Q may have a third set of servers 312 (e.g., 1 to F). Data center 310-2 may further comprise private cloud server 360-2 communicatively coupled to all servers 312-1 to 312-N in data center 310-2. Private cloud server 360-2 may be communicatively coupled to external cloud servers 210 in cloud 4.
Groups 340 may be logical groups, discussed in greater detail with respect to
Referring to
Cloud integration service 210-1 may include data monitoring 214 and cloud services 216. Dell CloudIQ is an example of a data monitoring service 214 that may monitor operation of servers 312 in data centers 310 for data protection. Dell APEX is an example catalog of cloud services 216 that may integrate infrastructure and software for cloud operations.
Infrastructure Solutions Group (ISG) server 210-2 may provide storage and networking services, and may include third-party services 350-18. In some embodiments, ISG server 210-2 may provide RAC proxy on Data Processing Unit (DPU) service 220 to private cloud server 360 to offload processing.
Management Service Catalog (MSC) server 210-3 may store a database of available services 350 that can be provided to private cloud servers 360. In some embodiments, private cloud server 360 communicates with MSC server 210-3 to retrieve available services 350. In some embodiments, MSC server 210-3 communicates with private cloud server 360 to install available services 350.
IOT integration service 210-4 may enable devices to communicate with other devices and enable external cloud servers 210 to communicate with or monitor devices in servers 312.
Autonomous server manager 210-5 may provide server management without a user.
Turning to one or more of
Firmware within RAC 314 on servers 312 may include a REpresentational State Transfer Application Programming Interface (REST API) 16 that allows servers 312 to communicate with each other. Remote monitoring may require communications with processes running in RAC 314. A D-Bus 18 or other inter-process communications mechanism enables local communication between processes on RAC 314. RAC 314 may further comprise Remote Procedure Call (RPC) queue 302, D-Bus to RPC mapper 304 and D-Bus to REST mapper 306 for available services 350 on private cloud server 360 to communicate with HAL 22. In some embodiments, RPC mapper 304 may be a Google RPC (gRPC) mapper 304.
RAC 314 may also store hardware abstraction layer (HAL) 22 for communication and monitoring of peripheral devices. Within HAL 22, I2C 22-1, I3C 22-2, SPI 22-3, RMII 22-4, PCIe VDM 22-5 and MCTP 22-6 allow communications with chips, processors and microcontrollers in server 312.
Embodiments of private cloud server 360 may store a set of available services 350 retrieved from MSC server 210-3. In some embodiments, private cloud server 360 may determine, based on one or more of an application to be executed on a server 312 or a data set to be processed by a server 312, that a particular available service is needed and communicate with MSC server 210-3 to retrieve the available service 350. In some embodiments, private cloud server 360 may communicate with MSC server 210-3 to retrieve an available service 350 based on a subscription. Available services 350 may be executed by private cloud server 360 unless there is a need to have RAC 314 execute the available service 350.
Embodiments may install available services 350 in RAC 314 from private cloud server 360, wherein the set of available services 350 stored in RAC 314 may be less than the total number of available services 350 stored in private cloud server 360 and less than the plurality of available services 350 stored in MSC server 210-3. Available services 350 may be installed in RAC 314 based on one or more of performance or capability. Once the process is complete or the available service 350 is no longer time-sensitive, embodiments may uninstall the available service 350 from RAC 314.
As a device-based example, referring to available service 350-13 (i.e., MCTP 350-13) and available service 350-10 (i.e., NC-SI 1.2 protocols 350-10), if a process sends a request to CPLD 26-6, a response may be expected in a minimum response time of about 50 milliseconds (ms) and a maximum response time of about 4 seconds. If the MCTP 350-13 and NC-SI 1.2 protocols 350-10 are stored in private cloud server 360, a response may take 5 seconds. Embodiments may install NC-SI 1.2 protocols 350-10 as a time sensitive plug-in 340 to ensure a response is received in less than 4 seconds. Once NC-SI 1.2 protocols 350-10 is not needed, NC-SI 1.2 protocols 350-10 may be uninstalled from RAC 314.
As another example, referring to available service 350-11 (i.e., PLDM 350-11) and available service 350-7 (i.e., Firmware update service 350-7), if there is a firmware update and PLDM 350-11 executing on private cloud server 360 cannot deliver the update payload within a maximum time (e.g., 4 seconds), the firmware update may time out. In this case, PLDM 350-11 and Firmware update service 350-7 may be installed as time sensitive plug-ins 340 to ensure the firmware update payload can be delivered in time. Once the firmware update payload is delivered, PLDM 350-11 and Firmware update service 350-7 may be uninstalled from RAC 314.
Box 352 contains a non-limiting list of available services 350 that may be installed on private cloud server 360 to communicate with RACs 314 on any servers 312 in data centers 310, wherein available services 350 in box 352 may function similar to features 30 described above with respect to
New Services and Services with Increased Scope
Embodiments may allow private cloud server 360 to provide additional available services 350 and available services 350 with increased scope that increase the capabilities of RAC 314. Box 354 contains a non-limiting list of available services 350 that may be retrieved from MSC server 210-3 and installed on private cloud server 360. Some available services 350 in box 354 may be executed by private cloud server 360. Some available services 350 in box 354 may be installed on RAC 314 on any server 312 in data center 310.
A user must have administrative credentials to run RACADM commands remotely. When a user wants to run RACADM commands, the user must first be validated by RAC 314. Traditionally, RAC 14 corresponding to server 12 in
Some Baseboard Management Controllers (BMCs) have no intelligence to adjust telemetry collection based on server events/errors or have different streaming rate for each metric. Traditionally, RAC 14 in server 12 may send the same information multiple times or at different rates, tying up networking resources and memory. Telemetry service 350-16 may be retrieved from MSC server 210-3 and installed on private cloud server 360. In some embodiments, an Open Telemetry (OTEL) service 350-16 may be stored in private cloud server 360 as an available service 350. When servers 312 communicate data for telemetry purposes, telemetry service 350-16 executing on private cloud server 360 may aggregate data, remove redundant data, or otherwise coordinate the communication of data, resulting in reduced network congestion. In some embodiments, telemetry service 350-16 may be installed in RAC 314 to meet telemetry requirements and then uninstalled after server 312 does not need to meet any telemetry requirements. In some embodiments, telemetry service 350-16 may be installed in RAC 314 as a time-sensitive plug-in 340 to provide quicker responses to telemetry requirements and then uninstalled after server 312 does not need quick responses to meet telemetry requirements.
Artificial Intelligence (AI)/Machine learning (ML) service 350-17 may include services necessary for AI/ML. If a server 312 (or a set of servers 312) is needed for AI/ML, AI/ML service 350-17 may be downloaded to private cloud server 360 for coordinating processing by servers 312 for AI/ML processing.
Third party services 350-18 may include services needed for particular third-party applications. Advantageously, instead of all available services 350 being tied to particular hardware, embodiments may enable third-party services 350-18 to execute on private cloud server 360, wherein other available services 350, RPC queue 302, D-Bus to RPC mapper 304 and D-Bus to REST mapper 306 enable third-party services 350-18 on private cloud server 360 to communicate with HAL 22 in one or more RACs 314.
Advanced power/thermal service 350-19 may be stored on private cloud server 360 and may refer to an available service 350 that can be executed on private cloud server 360 to communicate with RACs 314 on multiple servers 312 to monitor or control power or thermal operations of one or more servers 312. For example, data center 310 may have multiple floors with hundreds of racks of servers 312. Each RAC 314 may communicate with sensors inside a server 312 for remotely and individually monitoring temperature of that server 312. Advanced power/thermal service 350-19 may allow a user to remotely and collectively monitor and manage temperatures for multiple servers 312, such as all servers 312 processing a set of information, all servers 312 in a rack or all servers 312 on a floor. In some embodiments, advanced power/thermal service 350-19 may be installed in RAC 314 (e.g., as a time-sensitive plug-in 340) for quicker response to power/thermal requirements and then uninstalled after server 312 does not need to operate under advanced power/thermal requirements.
Secure Zero Touch Provisioning (sZTP) Service
Secure Zero-Touch Provisioning (sZTP) service 350-20 enables a server 312 to securely provision a network device when booting in a default state. If server 312 is expected to require communication over a network, sZTP service 350-20 executing on private cloud server 360 may ensure the network device is securely provisioned. In some embodiments, sZTP service 350-20 may be installed in RAC 314 for booting and uninstalled once server 312 has successfully booted. In some embodiments, sZTP service 350-20 may be installed on private cloud server 360 and installed on servers 312 (e.g., as a time-sensitive plug-in 340) as needed to ensure network devices are securely provisioned.
Integrated Dell Remote Access Controller (iDRAC) Service Manager service 350-21 may refer to available services 350-21 that may be executed to monitor and manage operation of RAC 314.
Fast IDentity Online (FIDO) Device Onboard (FDO) service 350-22 allows onboarding and trusting of new devices (e.g., RAC 314) within an environment. For example, RAC 314 may be Linux-based and one or more available services 350 may be Windows-based. FDO service 350-22 may enable servers 312 to execute Linux-based services 350 and Windows-based services 350. Data centers 310 may have hundreds or thousands of servers 312. Devices in servers 312 may be removed and exchanged for newer devices. Instead of a remote user configuring each device and deploying required applications, FDO service 350-22 allows multiple devices in various servers 312 to be configured correctly. In some embodiments, FDO service 350-22 may be installed in RAC 314 for quicker configuring and deployment and uninstalled once server 312 has successfully configured and deployed new devices. In some embodiments, FDO service 350-22 may be installed on private cloud server 360 and installed on servers 312 (e.g., as a time-sensitive plug-in 340) as needed to ensure new devices are quickly configured and deployed.
IOT Integration service 350-23 may be installed on private cloud server 360 to facilitate integration between devices on servers 312. Advantageously, instead of each server 312 in multiple data centers 310 communicating with IOT integration server 210-4, private cloud server 360 may perform some of the integration, wherein other available services 350, RPC queue 302, D-Bus to RPC mapper 304 and D-Bus to REST mapper 306 enable private cloud server 360 to communicate with HAL 22 in one or more RACs 314 in servers 312 that may have different devices.
Feature Tour service 350-24 may be installed to guide users through features available to server 312. Advantageously, instead of each RAC 314 in each server 312 in multiple data centers 310 storing all the information necessary to guide users through only features installed on that server, private cloud server 360 may store all the information and provide information including features and available services 350 available to a particular server 312. An end user may not know whether a feature is installed on server 312 or is instead an available service 350 accessible by server 312. In some embodiments, private cloud server 360 may install feature tour service 350-24 with information relevant to a particular server 312 on RAC 314 associated with the particular server 312.
IPDK service 350-25 is an open-source, vendor-agnostic framework of drivers and APIs. IPDK service 350-25 may be too large to install on each RAC 314 in each server 312. IPDK service 350-25 may be installed on private cloud server 360 for infrastructure offload and management.
For each available service 350, if the available service 350 is needed for a particular server 312, a version of the available service 350 may be retrieved from external cloud server 210 and executed on private cloud server 360, wherein RAC 314 contains a minimum number of built-in controls 20 and features 30 to communicate with private cloud server 360, wherein private cloud server 360 comprises memory and processors for executing the available service 350. If, at some later time, the available service 350 is not needed on a particular server 312, the available service 350 may be uninstalled from RAC 314 but a version may still be stored in private cloud server 360 or the available service 350 may be uninstalled from private cloud server 360.
MSO 356 may coordinate between features and ensure any service 350 is compatible with other available services 350 and hardware in server 312 and that no conflicts exist. For example, regarding telemetry, a manufacturer may have a default telemetry service 20 installed in firmware, but a customer may want to use Open Telemetry (OTel) service 350-16 or a third-party telemetry service 350-18. If the customer requests another telemetry service, MSO 356 may determine whether the requested telemetry application will work, if an existing telemetry feature needs to be disabled or an existing telemetry service 350-16 or 350-18 needs to be uninstalled.
Large Send Offload (LSO) service 358 may increase the egress throughput of high-bandwidth network connections. LSO service 358 may be installed on PCS 312, reducing workload performed by RAC 314. In some embodiments, LSO 358 enables communication with RAC Proxy on DPU service 220 to offload large workloads.
Turning to
The storage node 602 can include a memory management module 620, a baseband management controller (BMC) 622, a network interface card (NIC) 624, and memory 626. The BMC 622 can include a network interface card (NIC) 627. The memory 626 can include memory modules 628a, 628b, . . . , 628n. In some examples, the memory modules 628 are non-volatile dual inline memory modules (NVDIMM) over fabric. In some examples, the storage node 602 is similar to, or includes, the server 12, 312 of
The compute node 604 can include a memory management module 630, a remote access controller (RAC) 632, a network interface card (NIC) 634, and memory 636. The compute node 604 can further include virtual machines (VM) 635a, 635b, . . . 635n. The RAC 632 can include a network interface card (NIC) 637. The memory 636 can include memory modules 638a, 638b, . . . , 638n. In some examples, the memory modules 638 are non-volatile dual inline memory modules (NVDIMM) over fabric. In some examples, the memory modules 638 are dual inline memory modules (DIMM). In some examples, the compute node 604 is similar to, or includes, the server 12, 312 of
The storage node 602 can be physical different and separate from the compute node 604.
The private cloud server 606 can include a remote access controller (RAC) 640. The RAC 640 can include a virtual machine (VM) memory management module 642, a memory failure report service 644, a local proxy 646, and a VM service 648. In some examples, the private cloud server 606 is similar to, or includes, the private cloud server 360 of
The storage node 602 can be in communication with the compute node 604. Specifically, the NIC 624 of the storage node 602 can be in communication with the NIC 634 of the computing node 604 over the host network 608. Additionally, the NIC 627 of the BMC 622 of the storage node 602 can be in communication with the NIC 637 of the RAC 532 of the compute node 604 over the management network 610.
The private cloud server 606 can be in communication with the storage node 602 and the compute node 604.
The memory management module 630 and/or the VM memory management module 642 can assign, for each virtual machine 635, one or more of the memory modules 628 to the respective virtual machine 635, at 702. For example, the virtual machine 635b can be assigned to the memory module 628a of the storage node 602. That is, the virtual machine 635b utilizes the memory module 628a for facilitating processing and execution of computational tasks by the virtual machine 635b.
The virtual machines 635 can access, over the communication channel between the NIC 624 of the storage node 602 and the NIC 634 of the compute node 604, the respective memory modules 628, at 704. For example, the virtual machine 635b can access, over the communication channel between the NIC 624 of the storage node 602 and the NIC 634 of the compute node 604, the memory module 628a.
The memory management module 630 and/or the VM memory management module 642 can detect, for the virtual machines 635, a loss of connectivity between the virtual machines 635 and the memory modules 628, at 706. Specifically, the memory management module 630 and/or the VM memory management module 642 can detect loss of connectivity of the communication channel between the NIC 624 of the storage node 602 and the NIC 634 of the compute node 604 over the host network 608. For example, the memory management module 630 detects, for the virtual machine 635b, a loss of connectivity between the virtual machine 635b and the memory module 628a. The loss of connectively of the communication channel between the NIC 624 of the storage node 602 and the NIC 634 of the compute node 604 over the host network 608 can result from one or more of failure of the host network 608, intermediate node/switch failure or mis-configuration, failed software and firmware upgrades or patches, security breaches/network hacking, and failure of the NIC 624 and/or the NIC 634.
In some examples, the RAC 632 can facilitate tracking of such loss of connectively, e.g., at a storage device in communication with the RAC 632.
In response to detection of the loss of connectivity of the virtual machines 635 with respective memory modules 628, the memory management module 620 and/or the VM memory management module 642 automatically triggers a save operation of the data at the memory modules 628. For example, in response to detection of loss of connectivity of the communication channel between the NIC 624 of the storage node 602 and the NIC 634 of the compute node 604 over the host network 608, the memory management module 620 triggers a save operation of the data at the memory module 628a that is accessed by the virtual machine 635b.
Further in response to detection of the loss of connectivity of the virtual machines 635 with respective memory modules 628, the VM memory management module 642 assigns memory modules 638 to respective virtual machines 635, at 708. Specifically, further in response to a loss of connectively of the communication channel between the NIC 624 of the storage node 602 and the NIC 634 of the compute node 604 over the host network 608, the VM memory management module 642 and/or the memory management module 630 assigns one of the memory modules 638 to the virtual machine 635b. For example, the VM memory management module 642 and/or the memory management module 630 assigns the memory module 638a to the virtual machine 635b. That is, in response to the loss of connectively of the communication channel between the NIC 624 of the storage node 602 and the NIC 634 of the compute node 604 over the host network 608, the virtual machine 635b utilizes the memory module 638a for facilitating processing and execution of computational tasks at the virtual machine 635b. In other words, local memory module 638a is assigned to the virtual machine 635b (e.g., by the VM memory management module 642).
Further in response to detection of the loss of connectivity of the virtual machines 635 with respective memory modules 628, the VM memory management module 642 establishes a secondary communication channel between the storage node 602 and the compute node 604, at 710. Specifically, the VM memory management module 642 establishes a secondary communication channel between the BMC 622 of the storage node 602 and the RAC 632 of the computing node 604 over the management network 610. In particular, the VM memory management module 642 establishes a secondary communication channel between the NIC 627 of the BMC 622 of the storage node 602 and the NIC 637 of the RAC 632 of the computing node 604 over the management network 610.
In some examples, the secondary communication channel is an integrated Dell remote access controller (iDRAC) service module (iSM). iSM is a lightweight software service that integrates the operating system (OS) features with the RAC 632 of the compute node 604.
Further in response to detection of the loss of connectivity of the virtual machines 635 with respective memory modules 628, the memory failure report service 644 of the RAC 640 of the private cloud server 606 notifies the BMC 622 of the storage node 602 that a data transfer is to be initiated of the data from the memory modules 628 to the memory modules 638 over the management network 610, described further herein.
Further in response to detection of the loss of connectivity of the virtual machines 635 with respective memory modules 628, the VM memory management module 642 facilitates transferring, over the secondary communication channel between the storage node 602 and the computing node 604, data stored at the memory modules 628 to the memory modules 638, at 712. Specifically, in response to a loss of connectively of the communication channel between the NIC 624 of the storage node 602 and the NIC 634 of the compute node 604 over the host network 608, the VM memory management module 642 facilitates transferring, over the secondary communication channel between the NIC 627 of the BMC 622 of the storage node 602 and the NIC 637 of the RAC 632 of the computing node 604 over the management network 610, data stored at the memory modules 628b to the memory module 638a that is assigned to the virtual machine 635a.
Thus, the memory management module 620 transfers data from the memory module 628b through the NIC 627 of the BMC 622 over the management network 610 to the NIC 637 of the RAC 632 and to the memory module 628a by the memory management module 630. In other words, the memory management module 620 transfers data from the memory module 628b over the secondary communication channel, and specifically, iSM, to the memory module 628a.
After transferring, over the secondary communication channel between the storage node 602 and the computing node 604, data stored at the memory modules 628 to the memory modules 638, the virtual machines 635 are mapped to respective memory modules 638. For example, the memory management module 630 maps the virtual machine 635a to the memory module 628b.
After transferring, over the secondary communication channel between the NIC 627 of the BMC 622 of the storage node 602 and the NIC 637 of the RAC 632 of the computing node 604 over the management network 610, data stored at the memory module 628b to the memory module 638a that is assigned to the virtual machine 635a, the virtual machine 635a is restarted, at 714. Specifically, once all data is transferred to the memory module 638a from the memory module 628b over the management network 610, the virtual machine 635a is restarted. Further, operation of the virtual machine 635a is maintained after restarting, at 716. That is, in view of the loss of connectivity of the communication channel between the NIC 624 of the storage node 602 and the NIC 634 of the compute node 604 over the host network 608, operation of the virtual machine 635a is maintained.
In some examples, the local memory 636 can include a combination of non-volatile memory modules and volatile memory modules. For example, the local memory 636 can include NVDIMM and DIMM. For example, the memory module 638a can include non-volatile memory (NVDIMM) and the memory module 638b can include volatile memory (DIMM).
In some examples, the memory management module 630 and/or the VM memory management module 642, when assigning memory modules 638 to respective virtual machines 635, can determine whether the memory modules 638 include non-volatile memory modules. For example, when assigning memory modules 638 to the virtual machine 635b, it is determined whether memory modules 638 include non-volatile memory modules. In some examples, the memory management module 630 and/or the VM memory management module 642 determines that the memory modules 638 includes at least one non-volatile memory module (NVDIMM). In response to determining that the memory modules 638 includes at least one non-volatile memory module (e.g., memory module 638a), the VM memory management module 642 and/or the memory management module 630 assigns the memory module 638a to the virtual machine 635b. Further, the VM memory management module 642 facilitates transferring, over the secondary communication channel between the NIC 627 of the BMC 622 of the storage node 602 and the NIC 637 of the RAC 632 of the computing node 604 over the management network 610, data stored at the memory modules 628b to the memory module 638a that is assigned to the virtual machine 635a.
In some examples, the memory management module 630 and/or the VM memory management module 642 determines that the non-volatile memory modules (NVDIMM) of the memory modules 638 is not available. In response to determining that the non-volatile memory module (NVDIMM) of the memory modules 638 is not available (e.g., memory module 638a), the VM memory management module 642 and/or the memory management module 630 assigns a volatile memory module (DIMM) of the memory modules 638 to the virtual machine 635b. For example, the VM memory management module 642 and/or the memory management module 630 assigns the memory module 638b (DIMM) to the virtual machine 635b. Further, the VM memory management module 642 facilitates transferring, over the secondary communication channel between the NIC 627 of the BMC 622 of the storage node 602 and the NIC 637 of the RAC 632 of the computing node 604 over the management network 610, data stored at the memory modules 628b to the memory module 638b that is assigned to the virtual machine 635a.
In some examples, the memory management module 630 and/or the VM memory management module 642 determines that the non-volatile memory modules (NVDIMM) and the volatile memory modules (DIMM) of the memory modules 638 are not available. In response to determining that the non-volatile memory module (NVDIMM) and the volatile memory modules (DIMM) of the memory modules 638 are not available, the VM memory management module 642 assigns additional memory modules (not shown) operating at an additional storage node (not shown) to the virtual machine 635b.
The above disclosed subject matter is to be considered illustrative, and not restrictive, and the appended claims are intended to cover all such modifications, enhancements, and other embodiments which fall within the true spirit and scope of the present disclosure. Thus, to the maximum extent allowed by law, the scope of the present disclosure is to be determined by the broadest permissible interpretation of the following claims and their equivalents, and shall not be restricted or limited by the foregoing detailed description.
Herein, “or” is inclusive and not exclusive, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A or B” means “A, B, or both,” unless expressly indicated otherwise or indicated otherwise by context. Moreover, “and” is both joint and several, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A and B” means “A and B, jointly or severally,” unless expressly indicated otherwise or indicated other-wise by context.
The scope of this disclosure encompasses all changes, substitutions, variations, alterations, and modifications to the example embodiments described or illustrated herein that a person having ordinary skill in the art would comprehend. The scope of this disclosure is not limited to the example embodiments described or illustrated herein. Moreover, although this disclosure describes and illustrates respective embodiments herein as including particular components, elements, features, functions, operations, or steps, any of these embodiments may include any combination or permutation of any of the components, elements, features, functions, operations, or steps described or illustrated anywhere herein that a person having ordinary skill in the art would comprehend. Furthermore, reference in the appended claims to an apparatus or system or a component of an apparatus or system being adapted to, arranged to, capable of, configured to, enabled to, operable to, or operative to perform a particular function encompasses that apparatus, system, component, whether or not it or that particular function is activated, turned on, or unlocked, as long as that apparatus, system, or component is so adapted, arranged, capable, configured, enabled, operable, or operative.
