Method for using BMC as proxy NVMeoF discovery controller to provide NVM subsystems to host

Information

  • Patent Grant
  • 11861168
  • Patent Number
    11,861,168
  • Date Filed
    Monday, November 16, 2020
    4 years ago
  • Date Issued
    Tuesday, January 2, 2024
    12 months ago
Abstract
A management device that may communicate with at least one devices is disclosed. The management device may include a communication logic to communicate with the devices over a communication channels about data associated with the devices. The management device may also include reception logic that may receive a query from a host. The query may request information from the management device about the devices. The management device may also include a transmission logic to send the data about the devices to the host. The host may be configured to send a message to the devices.
Description
FIELD

The inventive concepts relate generally to Non-Volatile Memory Express Over Fabric (NVMeoF) systems, and more particularly to using a proxy for the host to discover installed NVMeoF devices.


BACKGROUND

Excerpts from the Non-Volatile Memory (NVM) Express (NVMe) over Fabrics (NVMeoF) specification 1.0, section 1.5.6, defines a discovery mechanism that a host may use to determine the NVM subsystems the host may access. A Discovery controller supports minimal functionality and only implements the required features that allow the Discovery Log Page to be retrieved. A Discovery controller does not implement Input/Output (I/O) queues or expose namespaces. A Discovery Service is an NVM subsystem that exposes only Discovery controllers. The method that a host uses to obtain the information necessary to connect to the initial Discovery Service is implementation specific.


The Discovery Log Page provided by a Discovery Controller contains one or more entries. Each entry specifies information necessary for the host to connect to an NVM subsystem via an NVMe Transport. An entry may specify an NVM subsystem that exposes namespaces that the host may access, or a referral to another Discovery Service. The maximum referral depth supported is eight levels.


The Baseboard Management Controller (BMC) has been widely used in servers, PCs, switches and other computer-based products. Generally speaking, the BMC depends on host processor and/or operating systems to initiate and complete the discovery process. In addition, the BMC does not care about what chassis it is in, since its main job is to monitor the health status of the system.


The sensors associated with the BMC measure internal physical variables such as temperature, humidity, power-supply voltage, fan speeds, communications parameters, and operating system (OS) functions. If any of these variables happens to stray outside specified limits, the administrator is notified. That person may then take corrective action by remote control. In some cases, the BMC may take some corrective actions such as increasing fan speeds or rebooting the failed subsystems. The monitored device/system may be power cycled or rebooted remotely, as necessary and/or appropriate. In this way, a single administrator may remotely manage numerous servers and other devices simultaneously, saving on the overall operating cost of the network and helping to ensure its reliability.


A need remains for a way for to reduce the time required for the host to identify all NVM devices in the chassis.





BRIEF DESCRIPTION OF THE DRAWINGS


FIG. 1 shows a chassis with a self-configuring Baseboard Management Controller (BMC) installed therein that may perform discovery of Non-Volatile Memory (NVM) devices, according to an embodiment of the inventive concept.



FIG. 2 shows additional details of the chassis of FIG. 1.



FIG. 3 shows the BMC of FIG. 1 communicating with devices on a mid-plane of the chassis of FIG. 1.



FIG. 4 shows details of the BMC of FIG. 1.



FIG. 5 shows details of the access logic of FIG. 4.



FIG. 6 shows the BMC of FIG. 1 with pins for signaling.



FIG. 7 shows the chassis of FIG. 1 in a High Availability configuration.



FIG. 8 shows the built-in self-configuration logic of FIG. 4.



FIG. 9 shows various sources for the drivers of FIG. 8.



FIG. 10 shows the device communication logic of FIG. 4.



FIG. 11 shows the host of FIG. 1 requesting a Discovery Log Page from the BMC of FIG. 1.



FIGS. 12A-12D show a flowchart of an example procedure for the BMC of FIG. 1 to self-configure, according to an embodiment of the inventive concept.



FIG. 13 shows a flowchart of an example procedure for access logic of FIG. 4 to determine the configuration of the chassis of FIG. 1.



FIG. 14 shows a flowchart of an example procedure for the BMC of FIG. 1 to perform discovery of NVM devices in the chassis of FIG. 1, according to an embodiment of the inventive concept.



FIG. 15 shows a flowchart of an example procedure for the device communication logic of FIG. 4 to obtain discovery information about the NVM devices of FIG. 3 in the chassis of FIG. 1.



FIG. 16 shows a flowchart of an example procedure for the BMC of FIG. 1 to build a record of the device(s) configurations.



FIG. 17 shows a flowchart of an example procedure for an NVM device in the chassis of FIG. 1 to inform the BMC of FIG. 1 about a change in the configuration of the NVM device, according to an embodiment of the inventive concept.





DETAILED DESCRIPTION

Reference will now be made in detail to embodiments of the inventive concept, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth to enable a thorough understanding of the inventive concept. It should be understood, however, that persons having ordinary skill in the art may practice the inventive concept without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.


It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first module could be termed a second module, and, similarly, a second module could be termed a first module, without departing from the scope of the inventive concept.


The terminology used in the description of the inventive concept herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the inventive concept. As used in the description of the inventive concept and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and/or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The components and features of the drawings are not necessarily drawn to scale.


U.S. patent application Ser. No. 15/256,495, filed Sep. 2, 2016, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/366,622, filed Jul. 26, 2016, both of which are incorporated by reference herein for all purposes, describes a self-discovery process by which Non-Volatile Memory (NVM) devices may perform self-discovery. This process may be extended to Baseboard Management Controllers (BMCs) that may perform self-discovery to get “Chassis Personality” information to complement self-configuring Solid-State Drives (SSDs).


The new BMC may perform the self-discovery process during boot up initialization. By reading “Chassis Personality” information from a known location of an Electrically Erasable Programmable Read Only Memory (EEPROM)—such as Vital Product Data (VPD) on the mid-plane—chassis-specific data may be obtained, and the BMC may respond appropriately. The BMC may discover, for example, whether it is in an NVM Express (NVMe) or NVMe over Fabric (NVMeoF) chassis. If the BMC is in an NVMeoF chassis, the BMC may enable appropriate NVMeoF functionalities such as Discovery Services, robust error reporting, and management capabilities, as well as multi-pathing BMCs in high availability configurations.


If the BMC self-discovery reveals that it is in an NVMe chassis, then the BMC may operate as a conventional BMC: i.e., no NMVeoF support. In NVMe mode, the drive discovery may be done through in-band PCI Express initialization/link training process. Thus, the new BMC may be used in both NVMe-based and NVMeoF-based systems.


In a large NMVeoF storage system, a BMC that may perform self-discovery may shorten the enumeration/discovery process significantly because:

    • All Network-attached SSD (NASSD) devices present in the system may perform self-discovery (as disclosed in U.S. patent application Ser. No. 15/256,495, filed Sep. 2, 2016, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/366,622, filed Jul. 26, 2016, both of which are incorporated by reference herein for all purposes) independently by reading from a known location from the system much quicker than a host CPU may.
    • The new BMC may perform self-discovery by reading from a known location for BMCs only, and be ready to behave appropriately in a much shorter period of time than that required by having a remote host/local processor ping/discover each device in the chassis, including the BMC.


Newer storage devices (or other devices, such as Network Interface Cards (NICs)) may use transport protocols such as NVMeoF to communicate with a chassis (also termed a host machine), and may support multiple transport protocols. When such devices are installed in a chassis, these devices may perform self-discovery during boot up and initialization. These devices may read VPD from a known location in an EEPROM: U.S. patent application Ser. No. 15/256,495, filed Sep. 2, 2016, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/366,622, filed Jul. 26, 2016, both of which are incorporated by reference herein for all purposes, describes such a self-discovery process.


Once self-discovery has started, these devices may then discover that they are installed in an NVMeoF chassis. These devices may then configure themselves to enable, for example, the Ethernet ports and disabling other unnecessary/unused/unsupported transport protocol support. In this way the operating system and host processor overhead related to multiple transport protocol discovery and management may be avoided.


In a large storage system, using such self-configuring devices may shorten the enumeration process significantly because all devices may perform self-discovery independently by reading from known location(s) from the system. The host processors and operating systems are not required to be present.


A BMC is a low-power controller embedded in servers or switches. A BMC may connect to sensors to read environmental conditions and to control devices. A BMC has all the connections to all NVMeoF devices via the control plane/path. Therefore, it is advantageous to use a BMC as a proxy for providing discovery services to the host or initiator. Due to its interaction with many devices, a BMC may serve as a Discovery Controller to provide a list of NVM subsystems that are accessible to the host.


The BMC presented herein may have firmware to perform discovery of eSSDs, Network-Attached Solid State Drives, or other devices inserted into the system. Network-Attached SSDs may include Ethernet SSDs, InfiniBand SSDs, Fibre-Channel SSDs, SSDs, or SSDs that offer a combination of these transport protocols (Ethernet, InfiniBand and, and Fibre-Channel). Ethernet, InfiniBand and, and Fibre-Channel transport protocols are merely exemplary, and embodiments of the inventive concept may include Network-Attached SSDs that support other transport protocols. The BMC may directly access each device through a private bus and a Complex Programmable Logic Device (CPLD). The BMC may also read a known non-volatile memory location on the mid-plane where each device reports its information. This method may shorten the enumeration process. The BMC may store each device's information as a Discovery Log Page in its non-volatile memory.


The BMC may communicate with devices using a control plane. The control plane, the data plane and the management plane are the three basic components of telecommunications products. The control plane is the part of a network that carries signaling traffic and is responsible for routing. Functions of the control plane include system configuration and management. The control plane and management plane serve the data plane, which bears the traffic that the network exists to carry. The management plane, which carries administrative traffic, is considered a subset of the control plane.


A new Intelligent Platform Management Interface (IPMI) command (System Discovery) may be supported by the BMC's firmware for local or remote hosts to retrieve this Discovery Log Page. Remote hosts may connect to the BMC through its Local Area Network (LAN) interface if they are in the same network. Remote hosts may also connect to the BMC's local host. Each entry in the Discovery Log Page may specify information necessary for the host to connect to an NVM subsystem via an NVMe Transport.


The NVMeoF standard specifies Discovery service may be performed via Ethernet links or via the data plane. In contrast, embodiments of the inventive concept use the BMC as a proxy, which enables discovery services to be performed via the control plane. In networking, the control plane is typically limited to only a system administrator, and is better protected than data plane, which may be accessed by many people/nodes. In terms of security, the control plane is better protected than data plane. In addition, system administrators may issue one command to a BMC to get all discovery log files from all NVMeoF devices instead of issuing one command per device, as specified by the standard.



FIG. 1 shows a chassis with a self-configuring Baseboard Management Controller (BMC) installed therein that may perform discovery of Non-Volatile Memory (NVM) devices, according to an embodiment of the inventive concept. In FIG. 1, chassis 105 is shown as a tower server, but chassis 105 may just as easily be a rack server.


Chassis 105 may include processor 110, memory 115, storage device 120, and BMC 125. Processor 110 may be any variety of processor: for example, an Intel Xeon, Celeron, Itanium, or Atom processor, an AMD Opteron processor, an ARM processor, etc. While FIG. 1 shows a single processor, chassis 105 may include any number of processors. Memory 115 may be any variety of memory, such as flash memory, Static Random Access Memory (SRAM), Persistent Random Access Memory, Ferroelectric Random Access Memory (FRAM), or Non-Volatile Random Access Memory (NVRAM), such as Magnetoresistive Random Access Memory (MRAM) etc., but is typically DRAM. Memory 115 may also be any desired combination of different memory types.


Storage device 120 may be any variety of storage device. Examples of such devices may include Solid State Drives (SSDs), but other storage forms, such as hard disk drives or other long-term storage devices, are also viable. BMC 125, as described above, may operate as a conventional BMC, but may also be self-configuring based on the configuration of chassis 105. For example, chassis 105 may be an NVMe chassis, or an NVMeoF chassis. With chassis 105 as an NVMe chassis, BMC 125 may operate as a conventional NVMe BMC after self-configuration. With chassis 105 as an NVMeoF chassis, BMC 125 may also operate as a conventional BMC, but it may also perform discovery of other devices within chassis 105, such as storage device 120, Network Interface Cards (NICs), and any other devices that may, like BMC 125, be subject to discovery.


While BMC 125 is described as being able to perform discovery of other devices in chassis 105, BMC 125 is one possible proxy for processor 110 performing the discovery. Other possible proxies may include a Redundant Array of Independent Disks (RAID) controller, another processor (typically different from processor 110, which would be involved in performing start-up operations), or even a software proxy. For the remainder of this document, any reference to BMC 125 is intended to also refer to these other proxy devices, as well as any other devices that may act as a proxy for processor 110.



FIG. 2 shows additional details of the chassis of FIG. 1. Referring to FIG. 2, typically, chassis 105 includes one or more processors 110, which may include memory controller 205 and clock 210, which may be used to coordinate the operations of the components of chassis 105. Processors 110 may also be coupled to memory 115, which may include random access memory (RAM), read-only memory (ROM), or other state preserving media, as examples. Processors 110 may also be coupled to storage devices 120, and to network connector 215, which may be, for example, an Ethernet connector or a wireless connector. Processors 110 may also be connected to a bus 220, to which may be attached user interface 225 and input/output interface ports that may be managed using input/output engine 230, among other components.



FIG. 3 shows BMC 125 of FIG. 1 communicating with devices on a mid-plane of chassis 105 of FIG. 1. In FIG. 3, BMC 125 and Complex Programmable Logic Device (CPLD) 305 may be situated on motherboard 310 within chassis 105 of FIG. 1. Chassis 105 of FIG. 1 may also include midplane 315. Midplane 315 may include other components, such as various Network-Attached SSDs 320, 325, and 330, which are examples of storage device 120 of FIG. 1. Network-Attached SSDs 320, 325, and 330 may support using any of a number of different transport protocols, such as Ethernet, Fibre Channel, InfiniBand, or Non-Volatile Memory Express (NVMe), to name a few possibilities, but in some embodiments of the inventive concept Network-Attached SSDs 320, 325, and/or 330 may be limited to a subset of these transport protocols (possibly one: for example, an Ethernet SSD). While FIG. 3 shows three Network-Attached SSDs 320, 325, and 330, embodiments of the inventive concept may support any desired number of devices. In addition, while FIG. 3 shows only Network-Attached SSDs 320, 325, and 330, other devices, such as Ethernet SSDs or NICs may be substituted for or included in addition to Network-Attached SSDs 320, 325, and 330. In the remainder of this document, any reference to Network-Attached SSDs 320, 325, and 330 is intended to encompass any alternative device that may be subject to discovery as an NVMeoF device and may be substituted for Network-Attached SSDs 320, 325, and 330.


BMC 125 may communicate with Network-Attached SSDs 320, 325, and 330 over I2C bus 335 and SMBus 340. Network-Attached SSDs 320, 325, and 330 may also communicate with EEPROM 345 and NVM 350. NVM 350 may act as memory 115 of FIG. 1; EEPROM 345 may store information for use by various devices in chassis 105 of FIG. 1. For example, EEPROM 345 may store VPD 355. VPD 355 may be used by Network-Attached SSDs 320, 325, and 330, and by BMC 125, to store information pertinent to those devices. More particularly, EEPROM 345 may store separate VPD 355 for each such device.


VPD 355 has several uses. In some embodiments of the inventive concept, VPD 355 may be used to store pertinent information for each device, which may be used in self-configuration. Thus, VPD 355 may store information used by Network-Attached SSDs 320, 325, and 330 to self-configure, as described in U.S. patent application Ser. No. 15/256,495, filed Sep. 2, 2016, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/366,622, filed Jul. 26, 2016, both of which are incorporated by reference herein for all purposes. But in other embodiments of the inventive concept, VPD 355 may also store information used by BMC 125 to perform its own self-configuration, as described below. In addition, in yet other embodiments of the inventive concept, Network-Attached SSDs 320, 325, and 330 may write information to VPD 355, which BMC 125 may then read. For example, Network-Attached SSDs 320, 325, and 330 may write their IP addresses to VPD 355, which BMC 125 may then read from VPD 355. Then, when host 110 of FIG. 1 queries BMC 125 for information, BMC 125 may provide the configuration information for Network-Attached SSDs 320, 325, and 330.


While FIG. 3 shows EEPROM 345 on midplane 315 and NVM 350 on motherboard 310, embodiments of the inventive concept may support these components (and other components as well) being placed anywhere desired. For example, in some embodiments of the inventive concept, EEPROM 345 and NVM 350 may both be located on midplane 315, in other embodiments of the inventive concept they may both be located on motherboard 310, and in yet other embodiments of the inventive concept NVM 350 may be on midplane 315 and EEPROM 345 on motherboard 310. Other embodiments of the inventive concept may place such components in yet other locations: for example, on another board within chassis 105 of FIG. 1, or possibly in another chassis entirely.



FIG. 4 shows details of BMC 125 of FIG. 1. In FIG. 4, BMC 125 is shown divided into two portions 405 and 410. Portion 405 relates to BMC 125 performing self-configuration in some embodiments of the inventive concept; portion 410 relates to BMC 125 acting as a proxy for host 110 of FIG. 1 in other embodiments of the inventive concept. Note that embodiments of the inventive concept may include one or both of portions 405 and 410, as desired.


To perform self-configuration, BMC 125 may include access logic 415, built-in self-configuration logic 420, and error reporting logic 425. Access logic 415 may access information about how BMC 125 is to configure itself. Access logic 415 is described further with reference to FIG. 5 below. Built-in self-configuration logic 420 may configure BMC 125 to use the appropriate driver based on the configuration of chassis 105 of FIG. 1. Built-in self-configuration logic 420 is described further with reference to FIG. 8 below. Error reporting logic 425 may report an error to host 110 of FIG. 1 when there is a problem. Examples of problems that BMC 125 might report to host 110 of FIG. 1 may include when chassis 105 of FIG. 1 is a High Availability chassis but BMC 125 may not access or load a High Availability driver, or when BMC 125 may not communicate with its pairing partner as a High Availability system.


To act as a discovery proxy for host 110 of FIG. 1, BMC 125 may include device communication logic 430, Log Page creation logic 435, reception logic 440, and transmission logic 445. Device communication logic 430 may enable BMC 125 to communicate with devices, such as Network-Attached SSDs 320, 325, and 330 of FIG. 3, to learn about their configuration. Device communication logic 430 is described further with reference to FIG. 10 below. Log Page creation logic 435 may take the information received from Ethernet devices 320, 325, and 330 of FIG. 3, and create a Discovery Log Page, that may be reported to host 110 of FIG. 1 at an appropriate time. Log Page creation logic 435 may either simply collate the information received from Ethernet devices 320, 325, and 330 of FIG. 3, or it may eliminate repeated information from Network-Attached SSDs 320, 325, and 330 of FIG. 3 by assembling the Log Page. The structure of a Log Page is described in the NVM Express over Fabrics specification, revision 1.0, dated Jun. 5, 2016, which is hereby incorporated by reference for all purposes.


In some embodiments of the inventive concept, BMC 125 may have its own storage: for example, in NVM 350 of FIG. 3, or in EEPROM 345 of FIG. 3 among other possibilities. Network-Attached SSDs 320, 325, and 330 of FIG. 3 may write their configuration information directly into a Log Page maintained in this storage for BMC 125.


Reception logic 440 and transmission logic 445 enable communication with host 110 of FIG. 1. For example, reception logic 440 may receive a query from host 110 of FIG. 1 regarding Network-Attached SSDs 320, 325, and 330 of FIG. 3; transmission logic 445 may send a response back to host 110 of FIG. 1 responsive to such a query. Note that reception logic 440 and transmission logic 445 are not required to be dedicated to the purposes described above: they may be used for other purposes as well. For example, as described below with reference to FIG. 10, device communication logic 430 may send messages to Network-Attached SSDs 320, 325, and 330 of FIG. 3: these messages may be sent using transmission logic 445 (and responses to these messages may be received using reception logic 440).



FIG. 5 shows details of access logic 415 of FIG. 4. In FIG. 5, access logic 415 may include VPD reading logic 505 and pin reading logic 510. VPD reading logic 505 may read information from VPD 355 of FIG. 3, which may be a VPD specific to BMC 125. The information in VPD 355 of FIG. 3 may include the configuration of chassis 105 of FIG. 1. Pin reading logic 510, on the other hand, may determine the configuration of chassis 105 of FIG. 1 by reading one or more signals on one or more pins of BMC 125 of FIG. 1. These pins of BMC 125 of FIG. 1 may be dedicated to specifying the configuration of chassis 105 of FIG. 1.



FIG. 6 shows an example of BMC 125 of FIG. 1 with pins for signaling. In FIG. 6, BMC 515 is shown as including a variety of pins. Pins 605 and 610 may be used to specify the configuration of chassis 105 of FIG. 1: based on the values signaled on these pins, BMC 125 may determine the configuration of chassis 105 of FIG. 1. Pins 605 and 610 may be general purpose input/output (GPIO) pins, among other possibilities.


Returning to FIG. 5, pin reading logic 510 may use the information read from pins 605 and 610 of FIG. 6 to determine the configuration of chassis 105 of FIG. 1 and load the appropriate driver. For example, as described below with reference to FIG. 8, there may be three different configurations of chassis 105 of FIG. 1: NVMe, NVMeoF, and High Availability. To choose between three different possibilities may require two bits, which could require signals to be sent on two pins. If the two pins specify the value 00, that combination may specify that chassis 105 of FIG. 1 is an NVMe chassis. If the two pins specify the value 01, that combination may specify that chassis 105 of FIG. 1 is an NVMeoF chassis. And if the two pins specify the value 10, that combination may specify that chassis 105 of FIG. 1 is a High Availability chassis.


Alternatively, three possibilities could be managed by a single pin. For example, a 0 value could specify an NVMe chassis, a 1 value could specify an NVMeoF chassis, and an oscillation between 0 and 1 could specify a High Availability chassis. But if there are more than three combinations, it is likely that more than one pin would be needed to specify the chassis configuration.


While the above example describes three possibilities—NVMe, NVMeoF, and High Availability—in other embodiments of the inventive concept there may be four driver configurations—NVMe, NVMeoF, NVMe High Availability, and NVMeoF High Availability. In such an embodiment of the inventive concept, for example, a high value on pin 605 may indicate that chassis 105 of FIG. 1 is a High Availability chassis and a low value on pin 605 may indicate that chassis 105 of FIG. 1 is not a High Availability chassis, whereas a high value on pin 610 may indicate that chassis 105 of FIG. 1 uses NVMeoF and a low value on pin 610 may indicate that chassis 105 of FIG. 1 uses NVMe. And in yet other embodiments of the inventive concept there may be even more different driver types. Embodiments of the inventive concept are may encompass any number of driver types as desired.


In FIG. 5, VPD reading logic 505 and pin reading logic 510 represent alternative ways for BMC 125 of FIG. 1 to determine the configuration of chassis 105 of FIG. 1. Thus, access logic 415 might include one or the other, and not necessarily both. However, embodiments of the inventive concept could include both VPD reading logic 505 and pin reading logic 510, to support BMC 125 of FIG. 1 being able to determine the configuration of chassis 105 of FIG. 1 in different ways.


High Availability chassis have now been mentioned a couple of times. FIG. 7 shows chassis 105 of FIG. 1 in a High Availability configuration. Processor 110 and BMC 125 may be paired with another processor 705 and another BMC 710. In some embodiments of the inventive concept, processor 705 may be in chassis 105 of FIG. 1, and in other embodiments of the inventive concept processor 705 may be in a different chassis. Processor 110 may maintain communication with processor 705, and BMC 125 may maintain communication with BMC 710. This communication may include a heartbeat: if one of BMC 125 and BMC 710 do not respond, then the other BMC knows that there is an error. Pairing partners may communicate, for example, over Peripheral Component Interconnect Express (PCIe) or Ethernet, among other possibilities.


If a pairing partner fails—for example, one of the chassis loses power—the remaining processor may enable the take-over path, permitting the remaining BMC to establish communication and cross the domain. Since the BMC in the failed chassis may run on standby power, the surviving processor may talk to the BMC of the failed chassis. The surviving processor may try to reset the failed processor, in the hopes that the failed processor may be restarted. If the failed processor may not be reset, the surviving processor may send an alert or interrupt to the host that oversees the failed chassis. A third party software or agent may then elect an available working node to become the new pairing partner of the surviving node.


Because of the need for heartbeat communication and for the surviving node to take over for the failed node, the driver needed for a High Availability chassis is different from the driver used in a non-High Availability chassis. Thus, BMC 125 operates differently in a High Availability chassis than in a non-High Availability chassis.


Until the High Availability driver is loaded into BMC 125, it might happen that BMC 125 may not see its pairing partner. Thus, in some embodiments of the inventive concept, the High Availability driver should be loaded even though BMC 125 might not yet be able to communicate with its pairing partner, and checking for the pairing partner should occur after the High Availability driver is loaded.



FIG. 8 shows built-in self-configuration logic 420 of FIG. 4. As described above, built-in self-configuration logic 420 may take the information determined by access logic 415 of FIG. 4 and configure BMC 125 of FIG. 1 accordingly. Built-in self-configuration logic 420 may include driver downloader 805 and driver loader 810. Driver downloader 805 may download an appropriate driver, such as NVMe driver 815, NVMeoF driver 820, and High Availability driver 825 from a driver source. Note that the driver source might be within the firmware of BMC 125 of FIG. 1, in which case the driver does not need to be “downloaded” at all, but rather just read from the firmware. Once downloaded or otherwise located, driver loader 810 may then load the selected driver into BMC 125.



FIG. 9 shows various sources for the drivers of FIG. 8. In FIG. 9, chassis 105 may include EEPROM 345, which may be a driver source. In such embodiments of the inventive concept, the appropriate driver may be stored in EEPROM 345 and read from there as needed.


Chassis 105 is also shown as connected to network 905. Network 905 may permit communication between chassis 105 and machines 910 and 915. Machine 910 may be a machine on a Local Area Network (LAN), whereas machine 915 may be a machine on a global network, such as the Internet. Regardless of what source exists for the selected driver, however, driver downloader 805 of FIG. 8 may download (or read) the appropriate driver from the driver source, enabling driver loader 810 of FIG. 8 to then load the driver into BMC 125 of FIG. 1.



FIG. 10 shows device communication logic 430 of FIG. 4. In FIG. 10, device communication logic 430 may include read logic 1005 and polling logic 1010. BMC 125 of FIG. 1 may use read logic 1005 to read information, such as the configuration of one of Network-Attached SSDs 320, 325, and 330 of FIG. 3, from VPD 355 of FIG. 3. BMC 125 may also use read logic 1005 to read new information from VPD 355, if Network-Attached SSDs 320, 325, and 330 send a message to BMC 125 of FIG. 1 indicating that new information is available.


In contrast, BMC 125 of FIG. 1 may using polling logic 1010 to poll Network-Attached SSDs 320, 325, and 330 of FIG. 3 periodically. In embodiments of the invention where Network-Attached SSDs 320, 325, and 330 of FIG. 3 do not notify BMC 125 of FIG. 1 about changes in their configuration, BMC 125 of FIG. 1 may use polling logic 1010 to query Network-Attached SSDs 320, 325, and 330 about their current information, and whether any information has changed. Network-Attached SSDs 320, 325, and 330 may then reply, indicating whether their configurations have changed and, if so, how they have changed.



FIG. 11 shows host 110 of FIG. 1 requesting a Discovery Log Page from BMC 125 of FIG. 1. In FIG. 11, host 110 may send query 1105 to BMC 125. Typically, query 1105 is sent when host 110 is ready to receive the configuration information of BMC 125, which may be some interval of time after BMC 125 has collected and assembled the configuration information for Network-Attached SSDs 320, 325, and 330. BMC 125 may then respond by with response 1110, which may include Log Page 1115. In this manner, BMC 125 may provide host 110 with information about all the devices installed in chassis 105, or at least, all the devices in the domain of the BMC. 125. For example, a single chassis 105 of FIG. 1 might have two motherboards (either two half-width motherboards or two stacked full-width motherboards, for example), each with its own BMC and attached devices. In such a scenario, each BMC is responsible for collecting the information about the devices in its domain, but not for collecting the information about the devices in the other BMC's domain, even though they are all within the same chassis.


Embodiments of the inventive concept have a technical advantage over conventional systems in that they may expedite the process of starting the machine. In conventional systems, the host must query each device in turn for its configuration information, but it may not do so until after it has done a number of other start-up operations. The BMC, in contrast, may start up much more quickly, and may act as a proxy for the host, querying the various devices for their configuration (while the host is busy performing other start-up procedures). Then, when the host is ready, the host may query the BMC for the configuration information, and may learn about all attached devices much more quickly. In addition, as compared with conventional Discovery Services performed using the data plane, performing discovery services via BMC 125 on the control plane is more secure and does not consume any bandwidth on the data plane.


Another technical advantage that embodiments of the inventive concept have over conventional systems is that the host only needs to issue one command to the BMC to perform discovery of all devices present in the chassis. For example, if the chassis includes 24 devices, the host may issue a “discovery all devices” command to the BMC: the BMC may discover the 24 devices. This approach avoids the host issuing 24 discovery commands to the 24 devices, as in the conventional system.



FIGS. 12A-12D show a flowchart of an example procedure for BMC 125 of FIG. 1 to self-configure, according to an embodiment of the inventive concept. In FIG. 12A, at block 1203, access logic 415 of FIG. 4 may determine a configuration of chassis 105 of FIG. 1: NVMe, NVMeoF, or High Availability. At block 1206, BMC 125 of FIG. 1 may determine if chassis 105 of FIG. 1 is a High Availability chassis (which may include multiple flavors, such as NVMe or NVMeoF). If chassis 105 of FIG. 1 is not a High Availability chassis, then processing may continue at block 1236 (FIG. 12C).


Continuing with FIG. 12A, if chassis 105 of FIG. 1 is a High Availability chassis, then at block 1209 built-in self-configuration logic 420 of FIG. 4 may select High Availability driver 825 of FIG. 8. At block 1212, built-in self-configuration logic 420 of FIG. 4 may determine if High Availability driver 825 of FIG. 8 is available. If not, then at block 1215, error reporting logic 425 of FIG. 4 may report an error.


If High Availability driver 825 of FIG. 8 is available, then at block 1218 (FIG. 12B) driver downloader 805 of FIG. 8 may download High Availability driver 825 of FIG. 8, and at block 1221 driver loader 810 of FIG. 8 may load High Availability driver 825 of FIG. 8.


At block 1224, BMC 125 of FIG. 1 may attempt to communicate with its pairing partner (BMC 710 of FIG. 7). At block 1227, BMC 125 of FIG. 1 may determine if its pairing partner is available. If the pairing partner of BMC 125 of FIG. 1 is available, then at block 1230, BMC 125 of FIG. 1 may determine whether the devices installed in chassis 105 of FIG. 1 are dual-path devices. If BMC 125 of FIG. 1 may not determine that its pairing partner is available (block 1227) or the devices installed in chassis 105 of FIG. 1 are not dual-path devices (block 1230), then at block 1233 BMC 125 of FIG. 1 may report that it is not operating as a High Availability device.


Regardless of whether chassis 105 of FIG. 1 is operating as a High Availability device (in blocks 1227, 1230, and 1233) or not (in block 1206), at block 1236, BMC 125 of FIG. 1 may determine if the configuration of chassis 105 of FIG. 1 is an NVMeoF chassis. If so, then at block 1239, BMC 125 of FIG. 1 may select NVMeoF driver 820 of FIG. 8, at block 1242 driver downloader 805 of FIG. 8 may download the NVMeoF driver 820 of FIG. 8, and at block 1245 driver loader 810 of FIG. 8 may load the NVMeoF driver 820 of FIG. 8. Additionally, at block 1248, BMC 125 of FIG. 1 may collect information about other devices installed in chassis 105 of FIG. 1, thereby acting as a proxy for host 110 of FIG. 1.


If chassis 105 of FIG. 1 is not an NVMeoF chassis, then at block 1251 (FIG. 12D), BMC 125 of FIG. 1 may determine if the configuration of chassis 105 of FIG. 1 is an NVMe chassis. If so, then at block 1254, BMC 125 of FIG. 1 may select NVMeoF driver 820 of FIG. 8, at block 1257 driver downloader 805 of FIG. 8 may download NVMe driver 815 of FIG. 8, and at block 1260 driver loader 810 of FIG. 8 may load NVMe driver 815 of FIG. 8, after which processing ends. If chassis 105 of FIG. 1 is not an NVMe chassis at block 1251, then control may return to block 1215 to report an error.



FIGS. 12A-12D show an example embodiment of the inventive concept. In other embodiments of the inventive concept, there may be more than two chassis configurations. And in yet other embodiments of the inventive concept, BMC 125 of FIG. 1 may use an NVMe driver as a default when no other driver may be loaded. Other variations on FIGS. 12A-12D are also possible.



FIG. 13 shows a flowchart of an example procedure for access logic 415 of FIG. 4 to determine the configuration of the chassis of FIG. 1. In FIG. 13, at block 1305, VPD reading logic 505 of FIG. 5 may read the configuration of chassis 105 of FIG. 1 from VPD 355 of FIG. 3. Alternatively, at block 1310, pin reading logic 510 may read the configuration of chassis 105 of FIG. 1 from signals sent on one or more pins 605 and 610 of FIG. 6 on BMC 125 of FIG. 1.



FIG. 14 shows a flowchart of an example procedure for BMC 125 of FIG. 1 to perform discovery of NVM devices in chassis 105 of FIG. 1, according to an embodiment of the inventive concept. In FIG. 14, at block 1405, BMC 125 may receive data about the configuration of Network-Attached SSDs 320, 325, and 330 of FIG. 3. Block 1405 may be repeated as often as necessary for all devices, as shown by dashed arrow 1410. At block 1415, BMC 125 of FIG. 1 may compile a record (such as Log Page 1115 of FIG. 11) from the information received from Network-Attached SSDs 320, 325, and 330 of FIG. 3. At block 1420, host 110 of FIG. 1 may send, and BMC 125 of FIG. 1 may receive, a request for the configurations about Network-Attached SSDs 320, 325, and 330 of FIG. 3. At block 1425, BMC 125 of FIG. 1 may send to host 110 of FIG. 1 the record of the device configurations.



FIG. 15 shows a flowchart of an example procedure for device communication logic 430 of FIG. 4 to obtain discovery information about NVM devices 320, 325, and 330 of FIG. 3 in chassis 105 of FIG. 1. In FIG. 15, at block 1505, read logic 1005 of FIG. 10 may read configuration data about a device from VPD 355 of FIG. 3. Alternatively, at block 1510, polling logic 1010 of FIG. 10 may poll the device for its configuration data, at block 1515 BMC 125 of FIG. 1 may receive the configuration data from the device.



FIG. 16 shows a flowchart of an example procedure for BMC 125 of FIG. 1 to build a record of the device(s) configurations. In FIG. 16, at block 1605, BMC 125 of FIG. 1 may simply compile the collected information from VPD 355 for the various devices into a record. Alternatively, at block 1610, Log Page creation logic 435 of FIG. 4 may create Log Page 1115 of FIG. 11 from the collected device(s) configurations.



FIG. 17 shows a flowchart of an example procedure for an NVM device in chassis 105 of FIG. 1 to inform BMC 125 of FIG. 1 about a change in the configuration of the NVM device, according to an embodiment of the inventive concept. In FIG. 17, at block 1705, the device—for example, Network-Attached SSDs 320, 325, and/or 330 of FIG. 3—may determine that its configuration has changed. At block 1710, the device may write the change to VPD 355, and at block 1715 the device may notify a proxy device—such as BMC 125 of FIG. 1—that the change was written to VPD 355. Alternatively, at block 1720, the device may wait until it receives a query from the proxy device about the device's current configuration, at which time (in block 1725) the device may send its current configuration to the proxy device.


In FIGS. 12A-17, some embodiments of the inventive concept are shown. But a person skilled in the art will recognize that other embodiments of the inventive concept are also possible, by changing the order of the blocks, by omitting blocks, or by including links not shown in the drawings. All such variations of the flowcharts are considered to be embodiments of the inventive concept, whether expressly described or not.


The following discussion is intended to provide a brief, general description of a suitable machine or machines in which certain aspects of the inventive concept may be implemented. The machine or machines may be controlled, at least in part, by input from conventional input devices, such as keyboards, mice, etc., as well as by directives received from another machine, interaction with a virtual reality (VR) environment, biometric feedback, or other input signal. As used herein, the term “machine” is intended to broadly encompass a single machine, a virtual machine, or a system of communicatively coupled machines, virtual machines, or devices operating together. Exemplary machines include computing devices such as personal computers, workstations, servers, portable computers, handheld devices, telephones, tablets, etc., as well as transportation devices, such as private or public transportation, e.g., automobiles, trains, cabs, etc.


The machine or machines may include embedded controllers, such as programmable or non-programmable logic devices or arrays, Application Specific Integrated Circuits (ASICs), embedded computers, smart cards, and the like. The machine or machines may utilize one or more connections to one or more remote machines, such as through a network interface, modem, or other communicative coupling. Machines may be interconnected by way of a physical and/or logical network, such as an intranet, the Internet, local area networks, wide area networks, etc. One skilled in the art will appreciate that network communication may utilize various wired and/or wireless short range or long range carriers and protocols, including radio frequency (RF), satellite, microwave, Institute of Electrical and Electronics Engineers (IEEE) 802.11, Bluetooth®, optical, infrared, cable, laser, etc.


Embodiments of the present inventive concept may be described by reference to or in conjunction with associated data including functions, procedures, data structures, application programs, etc. which when accessed by a machine results in the machine performing tasks or defining abstract data types or low-level hardware contexts. Associated data may be stored in, for example, the volatile and/or non-volatile memory, e.g., RAM, ROM, etc., or in other storage devices and their associated storage media, including hard-drives, floppy-disks, optical storage, tapes, flash memory, memory sticks, digital video disks, biological storage, etc. Associated data may be delivered over transmission environments, including the physical and/or logical network, in the form of packets, serial data, parallel data, propagated signals, etc., and may be used in a compressed or encrypted format. Associated data may be used in a distributed environment, and stored locally and/or remotely for machine access.


Embodiments of the inventive concept may include a tangible, non-transitory machine-readable medium comprising instructions executable by one or more processors, the instructions comprising instructions to perform the elements of the inventive concepts as described herein.


Having described and illustrated the principles of the inventive concept with reference to illustrated embodiments, it will be recognized that the illustrated embodiments may be modified in arrangement and detail without departing from such principles, and may be combined in any desired manner. And, although the foregoing discussion has focused on particular embodiments, other configurations are contemplated. In particular, even though expressions such as “according to an embodiment of the inventive concept” or the like are used herein, these phrases are meant to generally reference embodiment possibilities, and are not intended to limit the inventive concept to particular embodiment configurations. As used herein, these terms may reference the same or different embodiments that are combinable into other embodiments.


The foregoing illustrative embodiments are not to be construed as limiting the inventive concept thereof. Although a few embodiments have been described, those skilled in the art will readily appreciate that many modifications are possible to those embodiments without materially departing from the novel teachings and advantages of the present disclosure. Accordingly, all such modifications are intended to be included within the scope of this inventive concept as defined in the claims.


Embodiments of the inventive concept may extend to the following statements, without limitation:


Statement 1. An embodiment of the inventive concept includes a Baseboard Management Controller (BMC), comprising:


an access logic to determine a configuration of a chassis; and


a built-in self-configuration logic to configure the BMC responsive to the configuration of the chassis,


wherein the BMC may self-configure without using any BIOS, device drivers, or operating systems.


Statement 2. An embodiment of the inventive concept includes a BMC according to statement 1, wherein the built-in self-configuration logic is operative to configure the BMC to use either a Non-Volatile Memory Express (NVMe) driver or a Non-Volatile Memory Express Over Fabric (NVMeoF) driver responsive to the configuration for the BMC.


Statement 3. An embodiment of the inventive concept includes a BMC according to statement 2, wherein using the NVMeoF driver enables the BMC to determine the configuration of at least one device in a chassis including the BMC.


Statement 4. An embodiment of the inventive concept includes a BMC according to statement 2, wherein the access logic includes a Vital Product Data (VPD) reading logic to read the configuration of the chassis from a VPD.


Statement 5. An embodiment of the inventive concept includes a BMC according to statement 4, wherein the VPD is stored in an Electrically Erasable Programmable Read Only Memory (EEPROM).


Statement 6. An embodiment of the inventive concept includes a BMC according to statement 2, wherein the access logic includes a pin reading logic to determine the configuration of the chassis from a signal on at least one pin on the BMC.


Statement 7. An embodiment of the inventive concept includes a BMC according to statement 2, wherein the built-in self-configuration logic includes a driver loader to load the NVMe driver or the NVMeoF driver responsive to the configuration of the chassis.


Statement 8. An embodiment of the inventive concept includes a BMC according to statement 7, wherein the built-in self-configuration logic further includes a driver downloader to download the NVMe driver or the NVMeoF driver from a driver source.


Statement 9. An embodiment of the inventive concept includes a BMC according to statement 8, wherein the driver source is drawn from a set including storage in an EEPROM, a first site on a local computer network, and a second site on a global computer network.


Statement 10. An embodiment of the inventive concept includes a BMC according to statement 2, wherein the access logic is operative to determine whether the configuration of the chassis includes a High Availability (HA) chassis.


Statement 11. An embodiment of the inventive concept includes a BMC according to statement 10, wherein the built-in self-configuration logic is operative to load an HA driver.


Statement 12. An embodiment of the inventive concept includes a BMC according to statement 11, wherein the built-in self-configuration logic is operative to load the HA driver before the BMC has determined whether a pairing partner is available.


Statement 13. An embodiment of the inventive concept includes a BMC according to statement 11, further comprising an error reporting logic to report an error if the HA driver is not available.


Statement 14. An embodiment of the inventive concept includes a BMC according to statement 10, further comprising an error reporting logic to report an error if the BMC may not communicate with a pairing partner.


Statement 15. An embodiment of the inventive concept includes a method, comprising:


determining by a Baseboard Management Controller (BMC) a configuration of a chassis including the BMC;


selecting a driver responsive to the configuration of the chassis; and


loading the selected driver,


wherein the BMC may self-configure without using any BIOS, device drivers, or operating systems.


Statement 16. An embodiment of the inventive concept includes a method according to statement 15, wherein:


the configuration of chassis is drawn from a set including a Non-Volatile Memory Express (NVMe) chassis and a Non-Volatile Memory Express Over Fabric (NVMeoF) chassis; and


selecting a driver responsive to the configuration of the chassis includes selecting one of an NVMe driver and an NVMeoF driver for the BMC according to the configuration of the chassis.


Statement 17. An embodiment of the inventive concept includes a method according to statement 16, wherein:


determining by a Baseboard Management Controller (BMC) a configuration of a chassis including the BMC includes determining by the BMC that the configuration of the chassis is the NVMeoF chassis; and


the method further comprises determining, by the BMC, the configuration of at least one device in the chassis including the BMC.


Statement 18. An embodiment of the inventive concept includes a method according to statement 16, wherein determining by a Baseboard Management Controller (BMC) a configuration of a chassis including the BMC includes reading the configuration of the chassis from a Vital Product Data (VPD).


Statement 19. An embodiment of the inventive concept includes a method according to statement 18, wherein the VPD is stored in an Electrically Erasable Programmable Read Only Memory (EEPROM).


Statement 20. An embodiment of the inventive concept includes a method according to statement 16, wherein determining by a Baseboard Management Controller (BMC) a configuration of a chassis including the BMC includes accessing a signal from at least one pin on the BMC to determine the configuration of the chassis.


Statement 21. An embodiment of the inventive concept includes a method according to statement 16, further comprising downloading the selected driver from a driver source.


Statement 22. An embodiment of the inventive concept includes a method according to statement 21, wherein the driver source is drawn from a set including storage in an EEPROM, a first site on a local computer network, and a second site on a global computer network.


Statement 23. An embodiment of the inventive concept includes a method according to statement 16, wherein determining by a Baseboard Management Controller (BMC) a configuration of a chassis includes determining by the BMC that the configuration of the chassis is a High Availability (HA) chassis.


Statement 24. An embodiment of the inventive concept includes a method according to statement 23, wherein selecting a driver for the BMC according to the configuration of the chassis includes selecting an HA driver.


Statement 25. An embodiment of the inventive concept includes a method according to statement 24, further comprising reporting an error if the HA driver is not available.


Statement 26. An embodiment of the inventive concept includes a method according to statement 24, further comprising attempting to communicate with a pairing partner for the BMC.


Statement 27. An embodiment of the inventive concept includes a method according to statement 26, further comprising reporting an error if the BMC may not communicate with the pairing partner.


Statement 28. An embodiment of the inventive concept includes a method according to statement 26, wherein attempting to communicate with a pairing partner for the BMC includes attempting to communicate with the pairing partner for the BMC after loading the HA driver.


Statement 29. An embodiment of the inventive concept includes article, comprising a tangible storage medium, the tangible storage medium having stored thereon non-transitory instructions that, when executed by a machine, result in:


determining by a Baseboard Management Controller (BMC) a configuration of a chassis including the BMC;


selecting a driver responsive to the configuration of the chassis; and


loading the selected driver,


wherein the BMC may self-configure without using any BIOS, device drivers, or operating systems.


Statement 30. An embodiment of the inventive concept includes an article according to statement 29, wherein:


the configuration of chassis is drawn from a set including a Non-Volatile Memory Express (NVMe) chassis and a Non-Volatile Memory Express Over Fabric (NVMeoF) chassis;


selecting a driver responsive to the configuration of the chassis includes selecting one of an NVMe driver and an NVMeoF driver for the BMC according to the configuration of the chassis.


Statement 31. An embodiment of the inventive concept includes an article according to statement 30, wherein:


determining by a Baseboard Management Controller (BMC) a configuration of a chassis including the BMC includes determining by the BMC that the configuration of the chassis is the NVMeoF chassis; and


the tangible storage medium having stored thereon further non-transitory instructions that, when executed by the machine, result in determining, by the BMC, the configuration of at least one device in the chassis including the BMC.


Statement 32. An embodiment of the inventive concept includes an article according to statement 30, wherein determining by a Baseboard Management Controller (BMC) a configuration of a chassis including the BMC includes reading the configuration of the chassis from a Vital Product Data (VPD).


Statement 33. An embodiment of the inventive concept includes an article according to statement 32, wherein the VPD is stored in an Electrically Erasable Programmable Read Only Memory (EEPROM).


Statement 34. An embodiment of the inventive concept includes an article according to statement 30, wherein determining by a Baseboard Management Controller (BMC) a configuration of a chassis including the BMC includes accessing a signal from at least one pin on the BMC to determine the configuration of the chassis.


Statement 35. An embodiment of the inventive concept includes an article according to statement 30, the tangible storage medium having stored thereon further non-transitory instructions that, when executed by the machine, result in downloading the selected driver from a driver source.


Statement 36. An embodiment of the inventive concept includes an article according to statement 35, wherein the driver source is drawn from a set including storage in an EEPROM, a first site on a local computer network, and a second site on a global computer network.


Statement 37. An embodiment of the inventive concept includes an article according to statement 30, wherein determining by a Baseboard Management Controller (BMC) a configuration of a chassis includes determining by the BMC that the the chassis is a High Availability (HA) chassis.


Statement 38. An embodiment of the inventive concept includes an article according to statement 37, wherein selecting a driver for the BMC according to the configuration of the chassis includes selecting an HA driver.


Statement 39. An embodiment of the inventive concept includes an article according to statement 38, the tangible storage medium having stored thereon further non-transitory instructions that, when executed by the machine, result in reporting an error if the HA driver is not available.


Statement 40. An embodiment of the inventive concept includes an article according to statement 38, the tangible storage medium having stored thereon further non-transitory instructions that, when executed by the machine, result in attempting to communicate with a pairing partner for the BMC.


Statement 41. An embodiment of the inventive concept includes an article according to statement 40, the tangible storage medium having stored thereon further non-transitory instructions that, when executed by the machine, result in reporting an error if the BMC may not communicate with the pairing partner.


Statement 42. An embodiment of the inventive concept includes an article according to statement 40, wherein attempting to communicate with a pairing partner for the BMC includes attempting to communicate with the pairing partner for the BMC after loading the HA driver.


Statement 43. An embodiment of the inventive concept includes a proxy device in a chassis, comprising:


a device communication logic to communicate with at least one device over a control plane about data regarding the at least one device;


a reception logic to receive a query from a host, the query requesting information about the at least one device; and


a transmission logic to send a response to the host, the response including data about the at least one device.


Statement 44. An embodiment of the inventive concept includes a proxy device according to statement 43, wherein the proxy device is drawn from a set including a Baseboard Management Controller (BMC), a Redundant Array of Independent Disks (RAID) controller, and a processor.


Statement 45. An embodiment of the inventive concept includes a proxy device according to statement 43, wherein the at least one device is drawn from a set including a storage device and a Network Interface Card (NIC).


Statement 46. An embodiment of the inventive concept includes a proxy device according to statement 43, wherein the device communication logic includes a read logic to read the data regarding the at least one device from a Vital Product Data for the at least one device.


Statement 47. An embodiment of the inventive concept includes a proxy device according to statement 46, wherein the Vital Product Data is stored in an Electrically Erasable Programmable Read Only Memory (EEPROM).


Statement 48. An embodiment of the inventive concept includes a proxy device according to statement 43, wherein:


the device communication logic includes a polling logic to poll the at least one device for the data regarding the at least one device; and


the reception logic is operative to receive the data regarding the at least one device from the at least one device.


Statement 49. An embodiment of the inventive concept includes a proxy device according to statement 43, wherein the chassis includes permanent storage associated with the proxy device in which the proxy device may create a Log Page from the data regarding the at least one device.


Statement 50. An embodiment of the inventive concept includes a proxy device according to statement 49, further comprising a Log Page creation logic to create a Log Page from the data about the at least one device.


Statement 51. An embodiment of the inventive concept includes a proxy device according to statement 49, wherein the transmission logic is operative to send the Log Page to the host responsive to the query.


Statement 52. An embodiment of the inventive concept includes a method, comprising:


receiving, at a proxy device, at least one data from at least one device about configurations of the at least one device, the data from the at least one device received over a control plane;


compiling the at least one data into a record;


receiving, at the proxy device, a query from a host for the configurations of the at least one device; and


sending the record from the proxy device to the host,


wherein the proxy device may receive the at least one data from the at least one device and compile the at least one data into a record before receiving the query from the host.


Statement 53. An embodiment of the inventive concept includes a method according to statement 52, wherein the proxy device is drawn from a set including a Baseboard Management Controller (BMC), a Redundant Array of Independent Disks (RAID) controller, a processor, or a software proxy device.


Statement 54. An embodiment of the inventive concept includes a method according to statement 52, wherein the at least one device is drawn from a set including a storage device and a Network Interface Card (NIC).


Statement 55. An embodiment of the inventive concept includes a method according to statement 52, wherein receiving, at a proxy device, at least one data from at least one device about configurations of the at least one device includes receiving, at the proxy device, the at least one data from the at least one device about configurations of the at least one device along a control plane.


Statement 56. An embodiment of the inventive concept includes a method according to statement 52, wherein receiving, at a proxy device, at least one data from at least one device about configurations of the at least one device includes polling the at least one device for the configurations of the at least one device.


Statement 57. An embodiment of the inventive concept includes a method according to statement 52, wherein receiving, at a proxy device, at least one data from at least one device about configurations of the at least one device includes receiving a datum from one of the at least one device when a configuration of the one of the at least one device changes.


Statement 58. An embodiment of the inventive concept includes a method according to statement 52, wherein receiving, at a proxy device, at least one data from at least one device about configurations of the at least one device includes reading the at least one data from at least one Vital Product Data.


Statement 59. An embodiment of the inventive concept includes a method according to statement 58, wherein the at least one Vital Product Data are stored in an Electrically Erasable Programmable Read Only Memory (EEPROM).


Statement 60. An embodiment of the inventive concept includes a method according to statement 52, wherein compiling the at least one data into a record includes creating a Log Page from the at least one data.


Statement 61. An embodiment of the inventive concept includes a method according to statement 60, wherein sending the configurations of the at least one device from the proxy device to the host includes sending the Log Page from the proxy device to the host.


Statement 62. An embodiment of the inventive concept includes a method according to statement 52, wherein sending the configurations of the at least one device from the proxy device to the host includes sending the at least one data from the proxy device to the host.


Statement 63. An embodiment of the inventive concept includes a method, comprising:


determining, by a device, a change in a configuration of the device; and


notifying a proxy device over a control plane about the change in the configuration of the device.


Statement 64. An embodiment of the inventive concept includes a method according to statement 63, wherein the proxy device is drawn from a set including a Baseboard Management Controller (BMC), a Redundant Array of Independent Disks (RAID) controller, a processor, or a software proxy device.


Statement 65. An embodiment of the inventive concept includes a method according to statement 63, wherein the at least one device is drawn from a set including a storage device and a Network Interface Card (NIC).


Statement 66. An embodiment of the inventive concept includes a method according to statement 63, wherein notifying a proxy device about the change in the configuration of the device includes writing the change in the configuration of the device to a Vital Product Data that may be read by the proxy device.


Statement 67. An embodiment of the inventive concept includes a method according to statement 66, wherein notifying a proxy device about the change in the configuration of the device further includes notifying the proxy device that the change in the configuration of the device was written to the Vital Product Data.


Statement 68. An embodiment of the inventive concept includes a method according to statement 63, wherein notifying a proxy device about the change in the configuration of the device includes:


receiving a query from the proxy device about a current status of the configuration of the device; and


sending a response to the proxy device including the change in the configuration of the device.


Statement 69. An embodiment of the inventive concept includes an embodiment of the inventive concept includes an article, comprising a tangible storage medium, the tangible storage medium having stored thereon non-transitory instructions that, when executed by a machine, result in:


receiving, at a proxy device, at least one data from at least one device about configurations of the at least one device, the data from the at least one device received over a control plane;


compiling the at least one data into a record;


receiving, at the proxy device, a query from a host for the configurations of the at least one device; and


sending the record from the proxy device to the host,


wherein the proxy device may receive the at least one data from the at least one device and compile the at least one data into a record before receiving the query from the host.


Statement 70. An embodiment of the inventive concept includes an article according to statement 69, wherein the proxy device is drawn from a set including a Baseboard Management Controller (BMC), a Redundant Array of Independent Disks (RAID) controller, a processor, or a software proxy device.


Statement 71. An embodiment of the inventive concept includes an article according to statement 69, wherein the at least one device is drawn from a set including a storage device and a Network Interface Card (NIC).


Statement 72. An embodiment of the inventive concept includes an article according to statement 69, wherein receiving, at a proxy device, at least one data from at least one device about configurations of the at least one device includes receiving, at the proxy device, the at least one data from the at least one device about configurations of the at least one device along a control plane.


Statement 73. An embodiment of the inventive concept includes an article according to statement 69, wherein receiving, at a proxy device, at least one data from at least one device about configurations of the at least one device includes polling the at least one device for the configurations of the at least one device.


Statement 74. An embodiment of the inventive concept includes an article according to statement 69, wherein receiving, at a proxy device, at least one data from at least one device about configurations of the at least one device includes receiving a datum from one of the at least one device when a configuration of the one of the at least one device changes.


Statement 75. An embodiment of the inventive concept includes an article according to statement 69, wherein receiving, at a proxy device, at least one data from at least one device about configurations of the at least one device includes reading the at least one data from at least one Vital Product Data.


Statement 76. An embodiment of the inventive concept includes an article according to statement 75, wherein the at least one Vital Product Data are stored in an Electrically Erasable Programmable Read Only Memory (EEPROM).


Statement 77. An embodiment of the inventive concept includes an article according to statement 69, wherein compiling the at least one data into a record includes creating a Log Page from the at least one data.


Statement 78. An embodiment of the inventive concept includes an article according to statement 77, wherein sending the configurations of the at least one device from the proxy device to the host includes sending the Log Page from the proxy device to the host.


Statement 79. An embodiment of the inventive concept includes an article according to statement 69, wherein sending the configurations of the at least one device from the proxy device to the host includes sending the at least one data from the proxy device to the host.


Statement 80. An embodiment of the inventive concept includes an embodiment of the inventive concept includes an article, comprising a tangible storage medium, the tangible storage medium having stored thereon non-transitory instructions that, when executed by a machine, result in:


determining, by a device, a change in a configuration of the device; and


notifying a proxy device over a control plane about the change in the configuration of the device.


Statement 81. An embodiment of the inventive concept includes an article according to statement 80, wherein the proxy device is drawn from a set including a Baseboard Management Controller (BMC), a Redundant Array of Independent Disks (RAID) controller, a processor, or a software proxy device.


Statement 82. An embodiment of the inventive concept includes an article according to statement 80, wherein the at least one device is drawn from a set including a storage device and a Network Interface Card (NIC).


Statement 83. An embodiment of the inventive concept includes an article according to statement 80, wherein notifying a proxy device about the change in the configuration of the device includes writing the change in the configuration of the device to a Vital Product Data that may be read by the proxy device.


Statement 84. An embodiment of the inventive concept includes an article according to statement 83, wherein notifying a proxy device about the change in the configuration of the device further includes notifying the proxy device that the change in the configuration of the device was written to the Vital Product Data.


Statement 85. An embodiment of the inventive concept includes an article according to statement 80, wherein notifying a proxy device about the change in the configuration of the device includes:


receiving a query from the proxy device about a current status of the configuration of the device; and


sending a response to the proxy device including the change in the configuration of the device.


Consequently, in view of the wide variety of permutations to the embodiments described herein, this detailed description and accompanying material is intended to be illustrative only, and should not be taken as limiting the scope of the inventive concept. What is claimed as the inventive concept, therefore, is all such modifications as may come within the scope and spirit of the following claims and equivalents thereto.

Claims
  • 1. A management device, comprising: a communication logic to communicate with at least one device over one or more communication channels about data associated with the at least one device;a reception logic to receive a query from a host, the query requesting information about the at least one device; anda transmission logic to send a response to the host, the response including the data,wherein the host is configured to send a message intended for the at least one device,wherein the management device acts as a proxy for the at least one device by sending the response to the host, andwherein the host is configured to communicate with the at least one device via a data plane.
  • 2. The management device according to claim 1, wherein the management device is configured to communicate with the at least one device over the one or more communication channels about the data before the reception logic receives the query from the host.
  • 3. The management device according to claim 1, further comprising a read logic to read the data from a product data for the at least one device.
  • 4. The management device according to claim 1, wherein: the management device further comprises a polling logic to poll the at least one device for the data; andthe reception logic is operative to receive the data from the at least one device.
  • 5. The management device according to claim 1, wherein: the management device is in a chassis;the chassis includes storage associated with the management device; andthe management device is configured to create a log page from the data in the storage.
  • 6. The management device according to claim 5, further comprising a log page creation logic to create the log page from the data about the at least one device.
  • 7. The management device according to claim 5, wherein the transmission logic is operative to send the log page to the host responsive to the query.
  • 8. The management device according to claim 1, wherein the management device is configured to communicate with the at least one device via a control plane.
  • 9. A method, comprising: accessing, by a management device, a data associated with at least one device;receiving, at the management device, after accessing the data, a query from a host for the data, the query sent to the at least one device; andsending the data from the management device to the host,wherein the host is configured to communicate with the at least one device via a data plane.
  • 10. The method according to claim 9, wherein the management device is drawn from a set including a Baseboard Management Controller (BMC), a Redundant Array of Independent Disks (RAID) controller, a processor, or a software device.
  • 11. The method according to claim 9, wherein the at least one device is drawn from a set including a storage device and a Network Interface Card (NIC).
  • 12. The method according to claim 9, wherein accessing, by a management device, a data associated with at least one device includes reading, by the management device, the data from a product data.
  • 13. The method according to claim 12, wherein reading, by the management device, the data from a product data includes reading, by the management device, the data from product data stored in an Electrically Erasable Programmable Read Only Memory (EEPROM).
  • 14. The method according to claim 9, wherein accessing, by a management device, a data associated with at least one device includes: polling the at least one device, by the management device, for the data; andreceiving the data at the management device from the at least one device.
  • 15. The method according to claim 9, wherein accessing, by a management device, a data associated with at least one device includes accessing a datum from the at least one device when a configuration of the at least one device changes.
  • 16. The method according to claim 9, wherein: the method further comprises compiling the data into a record; andsending the data from the management device to the host includes sending the record from the management device to the host.
  • 17. The method according to claim 16, wherein compiling the data into a record includes creating a log page from the data.
  • 18. The method according to claim 17, wherein sending the data from the management device to the host includes sending the log page from the management device to the host.
  • 19. A system, comprising: a chassis;a processor in the chassis;a device; anda management device, including: a communication logic to communicate with the device over a communication channel about data associated with the device;a reception logic to receive a query from the processor, the query requesting information about the device; anda transmission logic to send a response to the processor, the response including the data,wherein the processor is configured to send a message intended for the device,wherein the management device acts as a proxy for the device by sending the response to a host, andwherein the host is configured to communicate with the device via a data plane.
  • 20. The system according to claim 19, wherein the management device is configured to communicate with the at least one device over the one or more communication channels about the data before the reception logic receives the query from the host.
RELATED APPLICATION DATA

This application is a continuation of U.S. patent application Ser. No. 16/424,474, filed May 28, 2019, now allowed, which is a continuation of U.S. patent application Ser. No. 15/345,507, filed Nov. 7, 2016, now U.S. Pat. No. 10,346,041, issued Jul. 9, 2019, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/394,726, filed Sep. 14, 2016, all of which are incorporated by reference herein for all purposes. This application is related to U.S. patent application Ser. No. 15/256,495, filed Sep. 2, 2016, now allowed, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/366,622, filed Jul. 26, 2016, both of which are incorporated by reference herein for all purposes. This application is related to U.S. patent application Ser. No. 15/345,509, filed Nov. 7, 2016, now pending, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/394,727, filed Sep. 14, 2016, both of which are incorporated by reference herein for all purposes.

US Referenced Citations (241)
Number Name Date Kind
6295567 Bassman et al. Sep 2001 B1
6345303 Knauerhase et al. Feb 2002 B1
6427198 Berglund et al. Jul 2002 B1
6611863 Banginwar Aug 2003 B1
6662119 Mitchell Dec 2003 B1
7107253 Sumner et al. Sep 2006 B1
7120759 Chiu Oct 2006 B2
7143153 Black et al. Nov 2006 B1
7249173 Nicolson Jul 2007 B2
7512585 Agarwal et al. Mar 2009 B2
7536486 Sadovsky et al. May 2009 B2
7620854 Kuttan et al. Nov 2009 B2
7873700 Pawlowski et al. Jan 2011 B2
7882393 Grimes et al. Feb 2011 B2
7944812 Carlson et al. May 2011 B2
8065347 DeMeyer et al. Nov 2011 B1
8180862 Baker et al. May 2012 B2
8396981 Lee et al. Mar 2013 B1
8599863 Davis Dec 2013 B2
8667224 Yu Mar 2014 B1
8754681 Zhu et al. Jun 2014 B2
8832327 Lin Sep 2014 B1
8943234 Voorhees et al. Jan 2015 B1
8949517 Cohen et al. Feb 2015 B2
8953644 Chandra et al. Feb 2015 B2
8998636 Gomez et al. Apr 2015 B2
9037786 Asnaashari et al. May 2015 B2
9047222 Chandra et al. Jun 2015 B2
9092321 Salessi Jul 2015 B2
9244865 Hutchison et al. Jan 2016 B2
9244877 Yang et al. Jan 2016 B2
9253275 Bhogal et al. Feb 2016 B2
9280357 Shaver et al. Mar 2016 B2
9280504 Ben-Michael et al. Mar 2016 B2
9389805 Cohen et al. Jul 2016 B2
9400749 Kuzmin et al. Jul 2016 B1
9460042 Iskandar et al. Oct 2016 B2
9465756 Bennett Oct 2016 B2
9648148 Rimmer et al. May 2017 B2
9653124 Heyd et al. May 2017 B2
9734093 Khemani et al. Aug 2017 B2
9734106 Kotzur et al. Aug 2017 B2
9785346 Yost Oct 2017 B2
9785355 Huang Oct 2017 B2
9785356 Huang Oct 2017 B2
9811481 Bhatia et al. Nov 2017 B2
9830082 Srinivasan et al. Nov 2017 B1
9842084 Friedman et al. Dec 2017 B2
9904330 Schuette et al. Feb 2018 B2
9906596 Sikdar Feb 2018 B2
9934173 Sakalley et al. Apr 2018 B1
9934183 Brassac et al. Apr 2018 B2
9959240 Mundt May 2018 B2
9965367 Shih May 2018 B2
9990313 Monji et al. Jun 2018 B2
10019388 Long et al. Jul 2018 B2
10063638 Huang Aug 2018 B2
10108450 Pinto et al. Oct 2018 B2
10114778 Worley et al. Oct 2018 B2
10162784 Bassett et al. Dec 2018 B2
10206297 Breakstone et al. Feb 2019 B2
10223313 Shih Mar 2019 B2
10223316 Mataya Mar 2019 B2
10235313 Lee et al. Mar 2019 B2
10255215 Breakstone et al. Apr 2019 B2
10275356 Chou et al. Apr 2019 B2
10289517 Beerens May 2019 B2
10289588 Chu et al. May 2019 B2
10318443 Su Jun 2019 B2
10346041 Olarig et al. Jul 2019 B2
10372648 Qiu Aug 2019 B2
10372659 Olarig et al. Aug 2019 B2
10394723 Yang et al. Aug 2019 B2
10452576 Stuhlsatz Oct 2019 B2
10467163 Malwankar et al. Nov 2019 B1
10467170 McKnight Nov 2019 B2
10474589 Raskin Nov 2019 B1
10560550 Xue et al. Feb 2020 B1
10592144 Roberts et al. Mar 2020 B2
10733137 Kachare et al. Aug 2020 B2
10866911 Qiu et al. Dec 2020 B2
10901927 Fischer et al. Jan 2021 B2
10929327 Schrempp et al. Feb 2021 B1
10942666 Pydipaty et al. Mar 2021 B2
11113046 Bowen et al. Sep 2021 B1
11126352 Olarig Sep 2021 B2
11347740 Moshe et al. May 2022 B2
20020087887 Busam et al. Jul 2002 A1
20020095491 Edmonds et al. Jul 2002 A1
20020123365 Thorson et al. Sep 2002 A1
20030058818 Wilkes et al. Mar 2003 A1
20040073912 Meza Apr 2004 A1
20040111590 Klein Jun 2004 A1
20040147281 Holcombe et al. Jul 2004 A1
20040153844 Ghose et al. Aug 2004 A1
20050025125 Kwan Feb 2005 A1
20050060442 Beverly et al. Mar 2005 A1
20050120157 Chen et al. Jun 2005 A1
20060059287 Rivard et al. Mar 2006 A1
20060095625 Wootten et al. May 2006 A1
20060098681 Cafiero et al. May 2006 A1
20060136621 Tung et al. Jun 2006 A1
20060202950 Lee et al. Sep 2006 A1
20070077553 Bentwich Apr 2007 A1
20080003845 Hong et al. Jan 2008 A1
20080288708 Hsueh Nov 2008 A1
20090073896 Gillingham et al. Mar 2009 A1
20090077478 Gillingham et al. Mar 2009 A1
20090217188 Alexander et al. Aug 2009 A1
20090222733 Basham et al. Sep 2009 A1
20090259364 Vollmer et al. Oct 2009 A1
20100077067 Strole Mar 2010 A1
20100100858 Schipper Apr 2010 A1
20100106836 Schreyer et al. Apr 2010 A1
20100169512 Matton et al. Jul 2010 A1
20110131380 Rallens Jun 2011 A1
20110151858 Lai Jun 2011 A1
20120056728 Erdmann et al. Mar 2012 A1
20120102580 Bealkowski Apr 2012 A1
20120207156 Srinivasan et al. Aug 2012 A1
20120311654 Dougherty, III et al. Dec 2012 A1
20120319750 Zhu et al. Dec 2012 A1
20130117503 Nellans et al. May 2013 A1
20130117766 Bax et al. May 2013 A1
20130179624 Lambert et al. Jul 2013 A1
20130198311 Tamir et al. Aug 2013 A1
20130198312 Tamir et al. Aug 2013 A1
20130242991 Basso et al. Sep 2013 A1
20130282953 Orme et al. Oct 2013 A1
20130304979 Zimmer et al. Nov 2013 A1
20130311795 Cong et al. Nov 2013 A1
20130318371 Hormuth Nov 2013 A1
20130325998 Hormuth et al. Dec 2013 A1
20140032641 Du Jan 2014 A1
20140052928 Shimoi Feb 2014 A1
20140122746 Shaver et al. May 2014 A1
20140195634 Kishore et al. Jul 2014 A1
20140195711 Bhatia et al. Jul 2014 A1
20140258679 McGee Sep 2014 A1
20140281458 Ravimohan Sep 2014 A1
20140317206 Lomelino et al. Oct 2014 A1
20140330995 Levy et al. Nov 2014 A1
20140344431 Hsu et al. Nov 2014 A1
20150006758 Holtman et al. Jan 2015 A1
20150039815 Klein Feb 2015 A1
20150067188 Chakhaiyar Mar 2015 A1
20150086017 Taylor et al. Mar 2015 A1
20150106660 Chumbalkar et al. Apr 2015 A1
20150120874 Kim et al. Apr 2015 A1
20150120971 Bae et al. Apr 2015 A1
20150138900 Choi May 2015 A1
20150178095 Balakrishnan et al. Jun 2015 A1
20150181760 Stephens Jun 2015 A1
20150205541 Nishtala et al. Jul 2015 A1
20150234815 Slik Aug 2015 A1
20150254088 Chou et al. Sep 2015 A1
20150255130 Lee et al. Sep 2015 A1
20150261434 Kagan et al. Sep 2015 A1
20150286599 Hershberger Oct 2015 A1
20150301757 Iwata Oct 2015 A1
20150301964 Brinicombe et al. Oct 2015 A1
20150304423 Satoyama et al. Oct 2015 A1
20150324312 Jacobson et al. Nov 2015 A1
20150331473 Jreji et al. Nov 2015 A1
20150350096 Dinc et al. Dec 2015 A1
20150370661 Swanson et al. Dec 2015 A1
20150370665 Cannata et al. Dec 2015 A1
20150376840 Shih Dec 2015 A1
20150381734 Ebihara et al. Dec 2015 A1
20160004879 Fisher et al. Jan 2016 A1
20160062936 Brassac et al. Mar 2016 A1
20160077841 Lambert et al. Mar 2016 A1
20160085718 Huang Mar 2016 A1
20160092390 Grothen et al. Mar 2016 A1
20160094619 Khan et al. Mar 2016 A1
20160127468 Malwankar et al. May 2016 A1
20160127492 Malwankar et al. May 2016 A1
20160146754 Prasad et al. May 2016 A1
20160147446 Ghosh May 2016 A1
20160188313 Dubal et al. Jun 2016 A1
20160246754 Rao et al. Aug 2016 A1
20160259597 Worley et al. Sep 2016 A1
20160261375 Roethig et al. Sep 2016 A1
20160283428 Guddeti Sep 2016 A1
20160306723 Lu Oct 2016 A1
20160306768 Mataya Oct 2016 A1
20160328344 Jose et al. Nov 2016 A1
20160328347 Worley et al. Nov 2016 A1
20160337272 Berman Nov 2016 A1
20160366071 Chandran et al. Dec 2016 A1
20170018149 Shih Jan 2017 A1
20170038804 Shows et al. Feb 2017 A1
20170063965 Grenader Mar 2017 A1
20170068268 Giriyappa et al. Mar 2017 A1
20170068628 Calciu et al. Mar 2017 A1
20170068630 Iskandar et al. Mar 2017 A1
20170168943 Chou et al. Jun 2017 A1
20170185554 Fricker Jun 2017 A1
20170187629 Shalev et al. Jun 2017 A1
20170206034 Fetik Jul 2017 A1
20170262029 Nelson et al. Sep 2017 A1
20170269871 Khan et al. Sep 2017 A1
20170270001 Suryanarayana et al. Sep 2017 A1
20170270060 Gupta et al. Sep 2017 A1
20170286305 Kalwitz Oct 2017 A1
20170317901 Agrawal et al. Nov 2017 A1
20170344259 Freyensee et al. Nov 2017 A1
20170344294 Mishra et al. Nov 2017 A1
20170357299 Shabbir et al. Dec 2017 A1
20170357515 Bower, III et al. Dec 2017 A1
20180004695 Chu et al. Jan 2018 A1
20180019896 Paquet et al. Jan 2018 A1
20180032463 Olarig et al. Feb 2018 A1
20180032469 Olarig et al. Feb 2018 A1
20180032471 Olarig Feb 2018 A1
20180052745 Marripudi et al. Feb 2018 A1
20180074717 Olarig et al. Mar 2018 A1
20180074984 Olarig et al. Mar 2018 A1
20180095904 Bunker et al. Apr 2018 A1
20180101492 Cho et al. Apr 2018 A1
20180131633 Li May 2018 A1
20180173652 Olarig et al. Jun 2018 A1
20180210517 Yun Jul 2018 A1
20180227369 DuCray et al. Aug 2018 A1
20180267925 Rees Sep 2018 A1
20180275919 Chirumamilla Sep 2018 A1
20180307650 Kachare et al. Oct 2018 A1
20180335958 Wu Nov 2018 A1
20180365185 Risinger Dec 2018 A1
20180373609 Beerens Dec 2018 A1
20190042424 Nair Feb 2019 A1
20190087268 Koltsidas et al. Mar 2019 A1
20190104632 Nelson et al. Apr 2019 A1
20190286584 Olarig et al. Sep 2019 A1
20190339888 Sasidharan et al. Nov 2019 A1
20200042217 Roberts et al. Feb 2020 A1
20200117663 Moshe et al. Apr 2020 A1
20200293916 Li Sep 2020 A1
20210342281 Olarig et al. Nov 2021 A1
20220188002 Olarig Jun 2022 A1
20220206693 Jung et al. Jun 2022 A1
Foreign Referenced Citations (38)
Number Date Country
1641568 Jul 2005 CN
101847429 May 2012 CN
103946824 Jul 2014 CN
104025063 Sep 2014 CN
104202197 Dec 2014 CN
104572516 Apr 2015 CN
104615577 May 2015 CN
105260275 Jan 2016 CN
105912275 Aug 2016 CN
103412769 Nov 2017 CN
2290497 Mar 2011 EP
2843557 Mar 2015 EP
2001290752 Oct 2001 JP
4257050 Apr 2009 JP
2010146525 Jul 2010 JP
2011048534 Mar 2011 JP
2012506184 Mar 2012 JP
2013041390 Feb 2013 JP
2014241545 Dec 2014 JP
2015049742 Mar 2015 JP
2015191649 Nov 2015 JP
2015194005 Nov 2015 JP
2015532985 Nov 2015 JP
2016037501 Mar 2016 JP
2016045968 Apr 2016 JP
WO2015194005 Apr 2017 JP
20090106469 Oct 2009 KR
20120135205 Dec 2012 KR
20150047785 May 2015 KR
20150071898 Jun 2015 KR
20160074659 Jun 2016 KR
201445325 Dec 2014 TW
2013077867 May 2013 WO
2014209764 Dec 2014 WO
2015049742 Apr 2015 WO
2015191649 Dec 2015 WO
2016037501 Mar 2016 WO
2016085016 Jun 2016 WO
Non-Patent Literature Citations (142)
Entry
Corrected Notice of Allowability for U.S. Appl. No. 15/256,495, dated Mar. 18, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 16/424,474, dated Feb. 22, 2021.
Notice of Allowance for U.S. Appl. No. 16/202,079, dated Jan. 27, 2021.
Notice of Allowance for U.S. Appl. No. 16/921,923, dated Feb. 18, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 16/424,474, dated Mar. 29, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 16/921,923, dated Apr. 1, 2021.
Final Office Action for U.S. Appl. No. 16/844,995, dated Mar. 29, 2021.
Notice of Allowance for U.S. Appl. No. 15/403,088, dated Oct. 22, 2018.
OC3D, “What is the New U.2 SSD Connection?,” (https://www.overclock3d.net/news/storage/what_is_the_new_u_2_ssd_connection/1), Jul. 2016, retrieved Apr. 12, 2021, 5 pages.
Office Action for U.S. Appl. No. 15/403,088, dated Jun. 7, 2018.
Wikipedia, “SATA Express,” (https://en.wikipedia.org/wiki/SATA_Express), retrieved Apr. 12, 2021, 6 pages.
Wikipedia, “U.2,” (https://en.wikipedia.org/wiki/U.2), retrieved Apr. 12, 2021, 2 pages.
Advisory Action for U.S. Appl. No. 15/256,495, dated Feb. 1, 2019.
Corrected Notice of Allowability for U.S. Appl. No. 16/202,079, dated Jul. 22, 2020.
Fang, Chin, “Using NVMe Gen3 PCIe SSD Cards in High-density Servers for High-performance Big Data Transfer Over Multiple Network Channels”, SLAC National Accelerator Laboratory, Stanford University, Stanford, California, Feb. 7, 2015, 17 pages.
Final Office Action for U.S. Appl. No. 15/256,495, dated Dec. 4, 2019.
Final Office Action for U.S. Appl. No. 15/256,495, dated Oct. 19, 2018.
Final Office Action for U.S. Appl. No. 15/345,509, dated Feb. 21, 2019.
Final Office Action for U.S. Appl. No. 15/411,962, dated Dec. 20, 2018.
Final Office Action for U.S. Appl. No. 16/424,474, dated May 1, 2020.
Notice of Allowance for U.S. Appl. No. 15/256,495, dated Mar. 5, 2020.
Notice of Allowance for U.S. Appl. No. 15/345,507, dated Feb. 19, 2019.
Notice of Allowance for U.S. Appl. No. 15/411,962, dated Mar. 18, 2019.
Notice of Allowance for U.S. Appl. No. 16/202,079, dated Jun. 1, 2020.
Notice of Allowance for U.S. Appl. No. 16/421,458, dated Apr. 15, 2020.
Notice of Allowance for U.S. Appl. No. 16/424,474, dated Jul. 15, 2020.
NVM Express over Fabrics specification Revision 1.0; NVM Express Inc.; Jun. 5, 2016. (Year 2016).
Office Action for U.S. Appl. No. 15/256,495, dated Jun. 14, 2019.
Office Action for U.S. Appl. No. 15/256,495, dated Mar. 29, 2018.
Office Action for U.S. Appl. No. 15/345,507, dated Dec. 3, 2018.
Office Action for U.S. Appl. No. 15/345,509, dated Apr. 29, 2020.
Office Action for U.S. Appl. No. 15/345,509, dated Nov. 29, 2019.
Office Action for U.S. Appl. No. 15/345,509, dated Sep. 10, 2018.
Office Action for U.S. Appl. No. 15/345,509, dated Sep. 28, 2020.
Office Action for U.S. Appl. No. 15/411,962, dated Aug. 10, 2018.
Office Action for U.S. Appl. No. 16/202,079, dated Aug. 22, 2019.
Office Action for U.S. Appl. No. 16/202,079, dated Mar. 4, 2020.
Office Action for U.S. Appl. No. 16/421,458, dated Dec. 30, 2019.
Office Action for U.S. Appl. No. 16/424,474, dated Feb. 3, 2020.
Office Action for U.S. Appl. No. 16/424,474, dated Oct. 15, 2019.
Office Action for U.S. Appl. No. 16/844,995, dated Sep. 4, 2020.
Office Action for U.S. Appl. No. 16/857,172, dated Oct. 8, 2020.
Office Action for U.S. Appl. No. 16/921,923, dated Oct. 28, 2020.
Corrected Notice of Allowability for U.S. Appl. No. 15/256,495, dated May 13, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 15/345,509, dated Jun. 15, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 16/921,923, dated May 26, 2021.
Notice of Allowance for U.S. Appl. No. 15/345,509, dated May 13, 2021.
Notice of Allowance for U.S. Appl. No. 16/202,079, dated May 14, 2021.
Notice of Allowance for U.S. Appl. No. 16/424,474, dated Apr. 30, 2021.
Notice of Allowance for U.S. Appl. No. 16/857,172, dated May 3, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 16/202,079, dated Jul. 12, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 16/921,923, dated Jul. 14, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 15/345,509, dated Aug. 11, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 15/345,509, dated Dec. 10, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 16/692,997, dated Jun. 18, 2020.
Corrected Notice of Allowability for U.S. Appl. No. 16/844,995, dated Dec. 9, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 16/950,624, dated Jul. 16, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 16/950,624, dated Jun. 10, 2021.
Final Office Action for U.S. Appl. No. 16/211,923, dated Aug. 19, 2019.
Final Office Action for U.S. Appl. No. 16/692,997, dated Mar. 26, 2020.
Final Office Action for U.S. Appl. No. 17/063,501, dated Nov. 2, 2021.
Final Office Action for U.S. Appl. No. 17/063,507, dated Nov. 22, 2021.
Notice of Allowance for U.S. Appl. No. 16/211,923, dated Sep. 13, 2019.
Notice of Allowance for U.S. Appl. No. 16/692,997, dated Jun. 1, 2020.
Notice of Allowance for U.S. Appl. No. 16/950,624, dated May 10, 2021.
OC3D, “What is the New U.2 SSD Connection?” OC3D News, Jul. 2016, (https://www.overclock3d.net/news/storage/what_is_the_new_u_2_ssd_connection/1), retrieved May 2018, 4 pages.
Office Action for U.S. Appl. No. 16/202,079, dated Dec. 9, 2021.
Office Action for U.S. Appl. No. 16/211,923, dated May 6, 2019.
Office Action for U.S. Appl. No. 16/692,997, dated Dec. 19, 2019.
Office Action for U.S. Appl. No. 16/950,624, dated Jan. 25, 2021.
Office Action for U.S. Appl. No. 17/063,501, dated Jul. 15, 2021.
Office Action for U.S. Appl. No. 17/063,507, dated Aug. 6, 2021.
SSD Form Factor Work Group, “Enterprise SSD Form Factor 1.0a”, 2012, SSD Form Factor Work Group, pp. 1-55. (Year: 2012).
Wikipedia, “SATA Express”, (https://en.wikipedia.org/wiki/SATA_Express), retrieved May 2018, 6 pages.
Wikipedia, “U.2”, (https://en.wikipedia.org/wiki/U.2), retreived May 2018, 3 pages.
Corrected Notice of Allowability for U.S. Appl. No. 15/345,509, dated Aug. 30, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 15/345,509, dated Oct. 27, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 16/202,079, dated Sep. 15, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 16/857,172, dated Aug. 20, 2021.
Corrected Notice of Allowability for U.S. Appl. No. 16/921,923, dated Aug. 24, 2021.
Notice of Allowance for U.S. Appl. No. 16/844,995, dated Sep. 29, 2021.
Office Action for U.S. Appl. No. 17/022,075, dated Oct. 15, 2021.
Notice of Allowance for U.S. Appl. No. 15/345,509, dated Feb. 8, 2022.
Notice of Allowance for U.S. Appl. No. 16/202,079, dated Mar. 17, 2022.
Office Action for U.S. Appl. No. 16/844,995, dated Feb. 22, 2022.
Office Action for U.S. Appl. No. 17/063,501, dated Feb. 24, 2022.
Office Action for U.S. Appl. No. 17/063,507, dated Mar. 17, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 15/345,509, dated Jun. 10, 2022.
Final Office Action for U.S. Appl. No. 17/022,075, dated May 25, 2022.
Notice of Allowance for U.S. Appl. No. 17/063,501, dated Jun. 2, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 15/345,509, dated Jun. 27, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 16/202,079, dated Jul. 7, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 17/063,501, dated Jul. 7, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 17/063,501, dated Jun. 23, 2022.
Final Office Action for U.S. Appl. No. 17/063,507, dated Jun. 27, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 15/345,509, dated Aug. 8, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 16/202,079, dated Aug. 17, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 17/063,501, dated Jul. 27, 2022.
Office Action for U.S. Appl. No. 17/408,365, dated Aug. 2, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 15/345,509, dated Sep. 7, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 16/202,079, dated Aug. 31, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 17/022,075, dated Sep. 6, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 17/063,501, dated Aug. 24, 2022.
Notice of Allowance for U.S. Appl. No. 16/844,995, dated Aug. 26, 2022.
Notice of Allowance for U.S. Appl. No. 17/022,075, dated Aug. 23, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 16/844,995, dated Oct. 20, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 17/063,501, dated Oct. 13, 2022.
Notice of Allowance for U.S. Appl. No. 17/063,507, dated Oct. 7, 2022.
Office Action for U.S. Appl. No. 17/230,989, dated Oct. 31, 2022.
Office Action for U.S. Appl. No. 17/376,145, dated Sep. 30, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 17/022,075, dated Dec. 5, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 17/063,501, dated Nov. 21, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 17/063,507, dated Nov. 30, 2022.
Final Office Action for U.S. Appl. No. 17/408,365, dated Dec. 27, 2022.
Corrected Notice of Allowability for U.S. Appl. No. 16/844,995, dated Jan. 19, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 17/063,507, dated Jan. 17, 2023.
Office Action for U.S. Appl. No. 16/202,079, dated Feb. 2, 2023.
Office Action for U.S. Appl. No. 17/868,734, dated Feb. 14, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 16/844,995, dated Mar. 15, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 16/844,995, dated Mar. 22, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 16/844,995, dated May 5, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 17/063,507, dated Mar. 1, 2023.
Final Office Action for U.S. Appl. No. 17/408,365, dated Apr. 18, 2023.
Office Action for U.S. Appl. No. 17/022,075, dated May 11, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 16/844,995, dated Jun. 30, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 16/844,995, dated Jun. 7, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 17/063,507, dated Jun. 1, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 17/063,507, dated Jun. 20, 2023.
Final Office Action for U.S. Appl. No. 17/230,989, dated Jun. 28, 2023.
Notice of Allowance for U.S. Appl. No. 17/376,145, dated Jun. 20, 2023.
Notice of Allowance for U.S. Appl. No. 17/868,734, dated Jul. 6, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 16/844,995, dated Sep. 5, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 17/063,507, dated Aug. 16, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 17/376,145, dated Aug. 28, 2023.
Supplemental Notice of Allowability for U.S. Appl. No. 17/868,734, dated Aug. 14, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 16/844,995, dated Oct. 12, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 17/063,507, dated Sep. 27, 2023.
Corrected Notice of Allowability for U.S. Appl. No. 17/376,145, dated Oct. 12, 2023.
Final Office Action for U.S. Appl. No. 17/230,989, dated Sep. 26, 2023.
Notice of Allowance for U.S. Appl. No. 17/408,365, dated Sep. 19, 2023.
Supplemental Notice of Allowability for U.S. Appl. No. 17/408,365, dated Sep. 28, 2023.
Supplemental Notice of Allowability for U.S. Appl. No. 17/868,734, dated Sep. 25, 2023.
Related Publications (1)
Number Date Country
20210072898 A1 Mar 2021 US
Provisional Applications (1)
Number Date Country
62394726 Sep 2016 US
Continuations (2)
Number Date Country
Parent 16424474 May 2019 US
Child 17099776 US
Parent 15345507 Nov 2016 US
Child 16424474 US