The inventive concepts relate generally to Non-Volatile Memory Express Over Fabric (NVMeoF) systems, and more particularly to using a proxy for the host to discover installed NVMeoF devices.
Excerpts from the Non-Volatile Memory (NVM) Express (NVMe) over Fabrics (NVMeoF) specification 1.0, section 1.5.6, defines a discovery mechanism that a host may use to determine the NVM subsystems the host may access. A Discovery controller supports minimal functionality and only implements the required features that allow the Discovery Log Page to be retrieved. A Discovery controller does not implement Input/Output (I/O) queues or expose namespaces. A Discovery Service is an NVM subsystem that exposes only Discovery controllers. The method that a host uses to obtain the information necessary to connect to the initial Discovery Service is implementation specific.
The Discovery Log Page provided by a Discovery Controller contains one or more entries. Each entry specifies information necessary for the host to connect to an NVM subsystem via an NVMe Transport. An entry may specify an NVM subsystem that exposes namespaces that the host may access, or a referral to another Discovery Service. The maximum referral depth supported is eight levels.
The Baseboard Management Controller (BMC) has been widely used in servers, PCs, switches and other computer-based products. Generally speaking, the BMC depends on host processor and/or operating systems to initiate and complete the discovery process. In addition, the BMC does not care about what chassis it is in, since its main job is to monitor the health status of the system.
The sensors associated with the BMC measure internal physical variables such as temperature, humidity, power-supply voltage, fan speeds, communications parameters, and operating system (OS) functions. If any of these variables happens to stray outside specified limits, the administrator is notified. That person may then take corrective action by remote control. In some cases, the BMC may take some corrective actions such as increasing fan speeds or rebooting the failed subsystems. The monitored device/system may be power cycled or rebooted remotely, as necessary and/or appropriate. In this way, a single administrator may remotely manage numerous servers and other devices simultaneously, saving on the overall operating cost of the network and helping to ensure its reliability.
A need remains for a way for to reduce the time required for the host to identify all NVM devices in the chassis.
Reference will now be made in detail to embodiments of the inventive concept, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth to enable a thorough understanding of the inventive concept. It should be understood, however, that persons having ordinary skill in the art may practice the inventive concept without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.
It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first module could be termed a second module, and, similarly, a second module could be termed a first module, without departing from the scope of the inventive concept.
The terminology used in the description of the inventive concept herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the inventive concept. As used in the description of the inventive concept and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and/or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The components and features of the drawings are not necessarily drawn to scale.
U.S. patent application Ser. No. 15/256,495, filed Sep. 2, 2016, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/366,622, filed Jul. 26, 2016, both of which are incorporated by reference herein for all purposes, describes a self-discovery process by which Non-Volatile Memory (NVM) devices may perform self-discovery. This process may be extended to Baseboard Management Controllers (BMCs) that may perform self-discovery to get “Chassis Personality” information to complement self-configuring Solid-State Drives (SSDs).
The new BMC may perform the self-discovery process during boot up initialization. By reading “Chassis Personality” information from a known location of an Electrically Erasable Programmable Read Only Memory (EEPROM)—such as Vital Product Data (VPD) on the mid-plane—chassis-specific data may be obtained, and the BMC may respond appropriately. The BMC may discover, for example, whether it is in an NVM Express (NVMe) or NVMe over Fabric (NVMeoF) chassis. If the BMC is in an NVMeoF chassis, the BMC may enable appropriate NVMeoF functionalities such as Discovery Services, robust error reporting, and management capabilities, as well as multi-pathing BMCs in high availability configurations.
If the BMC self-discovery reveals that it is in an NVMe chassis, then the BMC may operate as a conventional BMC: i.e., no NMVeoF support. In NVMe mode, the drive discovery may be done through in-band PCI Express initialization/link training process. Thus, the new BMC may be used in both NVMe-based and NVMeoF-based systems.
In a large NMVeoF storage system, a BMC that may perform self-discovery may shorten the enumeration/discovery process significantly because:
Newer storage devices (or other devices, such as Network Interface Cards (NICs)) may use transport protocols such as NVMeoF to communicate with a chassis (also termed a host machine), and may support multiple transport protocols. When such devices are installed in a chassis, these devices may perform self-discovery during boot up and initialization. These devices may read VPD from a known location in an EEPROM: U.S. patent application Ser. No. 15/256,495, filed Sep. 2, 2016, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/366,622, filed Jul. 26, 2016, both of which are incorporated by reference herein for all purposes, describes such a self-discovery process.
Once self-discovery has started, these devices may then discover that they are installed in an NVMeoF chassis. These devices may then configure themselves to enable, for example, the Ethernet ports and disabling other unnecessary/unused/unsupported transport protocol support. In this way the operating system and host processor overhead related to multiple transport protocol discovery and management may be avoided.
In a large storage system, using such self-configuring devices may shorten the enumeration process significantly because all devices may perform self-discovery independently by reading from known location(s) from the system. The host processors and operating systems are not required to be present.
A BMC is a low-power controller embedded in servers or switches. A BMC may connect to sensors to read environmental conditions and to control devices. A BMC has all the connections to all NVMeoF devices via the control plane/path. Therefore, it is advantageous to use a BMC as a proxy for providing discovery services to the host or initiator. Due to its interaction with many devices, a BMC may serve as a Discovery Controller to provide a list of NVM subsystems that are accessible to the host.
The BMC presented herein may have firmware to perform discovery of eSSDs, Network-Attached Solid State Drives, or other devices inserted into the system. Network-Attached SSDs may include Ethernet SSDs, InfiniBand SSDs, Fibre-Channel SSDs, SSDs, or SSDs that offer a combination of these transport protocols (Ethernet, InfiniBand, and Fibre-Channel). Ethernet, InfiniBand, and Fibre-Channel transport protocols are merely exemplary, and embodiments of the inventive concept may include Network-Attached SSDs that support other transport protocols. The BMC may directly access each device through a private bus and a Complex Programmable Logic Device (CPLD). The BMC may also read a known non-volatile memory location on the mid-plane where each device reports its information. This method may shorten the enumeration process. The BMC may store each device's information as a Discovery Log Page in its non-volatile memory.
The BMC may communicate with devices using a control plane. The control plane, the data plane and the management plane are the three basic components of telecommunications products. The control plane is the part of a network that carries signaling traffic and is responsible for routing. Functions of the control plane include system configuration and management. The control plane and management plane serve the data plane, which bears the traffic that the network exists to carry. The management plane, which carries administrative traffic, is considered a subset of the control plane.
A new Intelligent Platform Management Interface (IPMI) command (System Discovery) may be supported by the BMC's firmware for local or remote hosts to retrieve this Discovery Log Page. Remote hosts may connect to the BMC through its Local Area Network (LAN) interface if they are in the same network. Remote hosts may also connect to the BMC's local host. Each entry in the Discovery Log Page may specify information necessary for the host to connect to an NVM subsystem via an NVMe Transport.
The NVMeoF standard specifies Discovery service may be performed via Ethernet links or via the data plane. In contrast, embodiments of the inventive concept use the BMC as a proxy, which enables discovery services to be performed via the control plane. In networking, the control plane is typically limited to only a system administrator, and is better protected than data plane, which may be accessed by many people/nodes. In terms of security, the control plane is better protected than data plane. In addition, system administrators may issue one command to a BMC to get all discovery log files from all NVMeoF devices instead of issuing one command per device, as specified by the standard.
Chassis 105 may include processor 110, memory 115, storage device 120, and BMC 125. Processor 110 may be any variety of processor: for example, an Intel Xeon, Celeron, Itanium, or Atom processor, an AMD Opteron processor, an ARM processor, etc. While
Storage device 120 may be any variety of storage device. Examples of such devices may include Solid State Drives (SSDs), but other storage forms, such as hard disk drives or other long-term storage devices, are also viable. BMC 125, as described above, may operate as a conventional BMC, but may also be self-configuring based on the configuration of chassis 105. For example, chassis 105 may be an NVMe chassis, or an NVMeoF chassis. With chassis 105 as an NVMe chassis, BMC 125 may operate as a conventional NVMe BMC after self-configuration. With chassis 105 as an NVMeoF chassis, BMC 125 may also operate as a conventional BMC, but it may also perform discovery of other devices within chassis 105, such as storage device 120, Network Interface Cards (NICs), and any other devices that may, like BMC 125, be subject to discovery.
While BMC 125 is described as being able to perform discovery of other devices in chassis 105, BMC 125 is one possible proxy for processor 110 performing the discovery. Other possible proxies may include a Redundant Array of Independent Disks (RAID) controller, another processor (typically different from processor 110, which would be involved in performing start-up operations), or even a software proxy. For the remainder of this document, any reference to BMC 125 is intended to also refer to these other proxy devices, as well as any other devices that may act as a proxy for processor 110.
BMC 125 may communicate with Network-Attached SSDs 320, 325, and 330 over I2C bus 335 and SMBus 340. Network-Attached SSDs 320, 325, and 330 may also communicate with EEPROM 345 and NVM 350. NVM 350 may act as memory 115 of
VPD 355 has several uses. In some embodiments of the inventive concept, VPD 355 may be used to store pertinent information for each device, which may be used in self-configuration. Thus, VPD 355 may store information used by Network-Attached SSDs 320, 325, and 330 to self-configure, as described in U.S. patent application Ser. No. 15/256,495, filed Sep. 2, 2016, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/366,622, filed Jul. 26, 2016, both of which are incorporated by reference herein for all purposes. But in other embodiments of the inventive concept, VPD 355 may also store information used by BMC 125 to perform its own self-configuration, as described below. In addition, in yet other embodiments of the inventive concept, Network-Attached SSDs 320, 325, and 330 may write information to VPD 355, which BMC 125 may then read. For example, Network-Attached SSDs 320, 325, and 330 may write their IP addresses to VPD 355, which BMC 125 may then read from VPD 355. Then, when host 110 of
While
To perform self-configuration, BMC 125 may include access logic 415, built-in self-configuration logic 420, and error reporting logic 425. Access logic 415 may access information about how BMC 125 is to configure itself. Access logic 415 is described further with reference to
To act as a discovery proxy for host 110 of
In some embodiments of the inventive concept, BMC 125 may have its own storage: for example, in NVM 350 of
Reception logic 440 and transmission logic 445 enable communication with host 110 of
Returning to
Alternatively, three possibilities could be managed by a single pin. For example, a 0 value could specify an NVMe chassis, a 1 value could specify an NVMeoF chassis, and an oscillation between 0 and 1 could specify a High Availability chassis. But if there are more than three combinations, it is likely that more than one pin would be needed to specify the chassis configuration.
While the above example describes three possibilities—NVMe, NVMeoF, and High Availability—in other embodiments of the inventive concept there may be four driver configurations—NVMe, NVMeoF, NVMe High Availability, and NVMeoF High Availability. In such an embodiment of the inventive concept, for example, a high value on pin 605 may indicate that chassis 105 of
In
High Availability chassis have now been mentioned a couple of times.
If a pairing partner fails—for example, one of the chassis loses power—the remaining processor may enable the take-over path, permitting the remaining BMC to establish communication and cross the domain. Since the BMC in the failed chassis may run on standby power, the surviving processor may talk to the BMC of the failed chassis. The surviving processor may try to reset the failed processor, in the hopes that the failed processor may be restarted. If the failed processor may not be reset, the surviving processor may send an alert or interrupt to the host that oversees the failed chassis. A third party software or agent may then elect an available working node to become the new pairing partner of the surviving node.
Because of the need for heartbeat communication and for the surviving node to take over for the failed node, the driver needed for a High Availability chassis is different from the driver used in a non-High Availability chassis. Thus, BMC 125 operates differently in a High Availability chassis than in a non-High Availability chassis.
Until the High Availability driver is loaded into BMC 125, it might happen that BMC 125 may not see its pairing partner. Thus, in some embodiments of the inventive concept, the High Availability driver should be loaded even though BMC 125 might not yet be able to communicate with its pairing partner, and checking for the pairing partner should occur after the High Availability driver is loaded.
Chassis 105 is also shown as connected to network 905. Network 905 may permit communication between chassis 105 and machines 910 and 915. Machine 910 may be a machine on a Local Area Network (LAN), whereas machine 915 may be a machine on a global network, such as the Internet. Regardless of what source exists for the selected driver, however, driver downloader 805 of
In contrast, BMC 125 of
Embodiments of the inventive concept have a technical advantage over conventional systems in that they may expedite the process of starting the machine. In conventional systems, the host must query each device in turn for its configuration information, but it may not do so until after it has done a number of other start-up operations. The BMC, in contrast, may start up much more quickly, and may act as a proxy for the host, querying the various devices for their configuration (while the host is busy performing other start-up procedures). Then, when the host is ready, the host may query the BMC for the configuration information, and may learn about all attached devices much more quickly. In addition, as compared with conventional Discovery Services performed using the data plane, performing discovery services via BMC 125 on the control plane is more secure and does not consume any bandwidth on the data plane.
Another technical advantage that embodiments of the inventive concept have over conventional systems is that the host only needs to issue one command to the BMC to perform discovery of all devices present in the chassis. For example, if the chassis includes 24 devices, the host may issue a “discovery all devices” command to the BMC: the BMC may discover the 24 devices. This approach avoids the host issuing 24 discovery commands to the 24 devices, as in the conventional system.
Continuing with
If High Availability driver 825 of
At block 1224, BMC 125 of
Regardless of whether chassis 105 of
If chassis 105 of
In
The following discussion is intended to provide a brief, general description of a suitable machine or machines in which certain aspects of the inventive concept may be implemented. The machine or machines may be controlled, at least in part, by input from conventional input devices, such as keyboards, mice, etc., as well as by directives received from another machine, interaction with a virtual reality (VR) environment, biometric feedback, or other input signal. As used herein, the term “machine” is intended to broadly encompass a single machine, a virtual machine, or a system of communicatively coupled machines, virtual machines, or devices operating together. Exemplary machines include computing devices such as personal computers, workstations, servers, portable computers, handheld devices, telephones, tablets, etc., as well as transportation devices, such as private or public transportation, e.g., automobiles, trains, cabs, etc.
The machine or machines may include embedded controllers, such as programmable or non-programmable logic devices or arrays, Application Specific Integrated Circuits (ASICs), embedded computers, smart cards, and the like. The machine or machines may utilize one or more connections to one or more remote machines, such as through a network interface, modem, or other communicative coupling. Machines may be interconnected by way of a physical and/or logical network, such as an intranet, the Internet, local area networks, wide area networks, etc. One skilled in the art will appreciate that network communication may utilize various wired and/or wireless short range or long range carriers and protocols, including radio frequency (RF), satellite, microwave, Institute of Electrical and Electronics Engineers (IEEE) 802.11, Bluetooth®, optical, infrared, cable, laser, etc.
Embodiments of the present inventive concept may be described by reference to or in conjunction with associated data including functions, procedures, data structures, application programs, etc. which when accessed by a machine results in the machine performing tasks or defining abstract data types or low-level hardware contexts. Associated data may be stored in, for example, the volatile and/or non-volatile memory, e.g., RAM, ROM, etc., or in other storage devices and their associated storage media, including hard-drives, floppy-disks, optical storage, tapes, flash memory, memory sticks, digital video disks, biological storage, etc. Associated data may be delivered over transmission environments, including the physical and/or logical network, in the form of packets, serial data, parallel data, propagated signals, etc., and may be used in a compressed or encrypted format. Associated data may be used in a distributed environment, and stored locally and/or remotely for machine access.
Embodiments of the inventive concept may include a tangible, non-transitory machine-readable medium comprising instructions executable by one or more processors, the instructions comprising instructions to perform the elements of the inventive concepts as described herein.
Having described and illustrated the principles of the inventive concept with reference to illustrated embodiments, it will be recognized that the illustrated embodiments may be modified in arrangement and detail without departing from such principles, and may be combined in any desired manner. And, although the foregoing discussion has focused on particular embodiments, other configurations are contemplated. In particular, even though expressions such as “according to an embodiment of the inventive concept” or the like are used herein, these phrases are meant to generally reference embodiment possibilities, and are not intended to limit the inventive concept to particular embodiment configurations. As used herein, these terms may reference the same or different embodiments that are combinable into other embodiments.
The foregoing illustrative embodiments are not to be construed as limiting the inventive concept thereof. Although a few embodiments have been described, those skilled in the art will readily appreciate that many modifications are possible to those embodiments without materially departing from the novel teachings and advantages of the present disclosure. Accordingly, all such modifications are intended to be included within the scope of this inventive concept as defined in the claims.
Embodiments of the inventive concept may extend to the following statements, without limitation:
Statement 1. An embodiment of the inventive concept includes a Baseboard Management Controller (BMC) (125), comprising:
an access logic (415) to determine a configuration of a chassis (105); and
a built-in self-configuration logic (420) to configure the BMC (125) responsive to the configuration of the chassis (105),
wherein the BMC (125) may self-configure without using any BIOS, device drivers, or operating systems.
Statement 2. An embodiment of the inventive concept includes a BMC (125) according to statement 1, wherein the built-in self-configuration logic (420) is operative to configure the BMC (125) to use either a Non-Volatile Memory Express (NVMe) driver (815) or a Non-Volatile Memory Express Over Fabric (NVMeoF) driver (820) responsive to the configuration for the BMC (125).
Statement 3. An embodiment of the inventive concept includes a BMC (125) according to statement 2, wherein using the NVMeoF driver (820) enables the BMC (125) to determine the configuration of at least one device (320, 325, 330) in a chassis (105) including the BMC (125).
Statement 4. An embodiment of the inventive concept includes a BMC (125) according to statement 2, wherein the access logic (415) includes a Vital Product Data (VPD) reading logic (505) to read the configuration of the chassis (105) from a VPD (355).
Statement 5. An embodiment of the inventive concept includes a BMC (125) according to statement 4, wherein the VPD (355) is stored in an Electrically Erasable Programmable Read Only Memory (EEPROM) (345).
Statement 6. An embodiment of the inventive concept includes a BMC (125) according to statement 2, wherein the access logic (415) includes a pin reading logic (510) to determine the configuration of the chassis (105) from a signal on at least one pin (605, 610) on the BMC (125).
Statement 7. An embodiment of the inventive concept includes a BMC (125) according to statement 2, wherein the built-in self-configuration logic (420) includes a driver loader (810) to load the NVMe driver (815) or the NVMeoF driver (820) responsive to the configuration of the chassis (105).
Statement 8. An embodiment of the inventive concept includes a BMC (125) according to statement 7, wherein the built-in self-configuration logic (420) further includes a driver downloader (805) to download the NVMe driver (815) or the NVMeoF driver (820) from a driver source (345, 910, 915).
Statement 9. An embodiment of the inventive concept includes a BMC (125) according to statement 8, wherein the driver source (345, 910, 915) is drawn from a set including storage in an EEPROM (345), a first site (910) on a local computer network (905), and a second site (915) on a global computer network (905).
Statement 10. An embodiment of the inventive concept includes a BMC (125) according to statement 2, wherein the access logic (415) is operative to determine whether the configuration of the chassis (105) includes a High Availability (HA) chassis (105).
Statement 11. An embodiment of the inventive concept includes a BMC (125) according to statement 10, wherein the built-in self-configuration logic (420) is operative to load an HA driver (825).
Statement 12. An embodiment of the inventive concept includes a BMC (125) according to statement 11, wherein the built-in self-configuration logic (420) is operative to load the HA driver (825) before the BMC (125) has determined whether a pairing partner (710) is available.
Statement 13. An embodiment of the inventive concept includes a BMC (125) according to statement 11, further comprising an error reporting logic (425) to report an error if the HA driver (825) is not available.
Statement 14. An embodiment of the inventive concept includes a BMC (125) according to statement 10, further comprising an error reporting logic (425) to report an error if the BMC (125) may not communicate with a pairing partner (710).
Statement 15. An embodiment of the inventive concept includes a method, comprising:
determining (1203) by a Baseboard Management Controller (BMC) (125) a configuration of a chassis (105) including the BMC (125);
selecting (1239, 1254) a driver (815, 820) responsive to the configuration of the chassis (105); and
loading (1245) the selected driver (815, 820),
wherein the BMC (125) may self-configure without using any BIOS, device drivers, or operating systems.
Statement 16. An embodiment of the inventive concept includes a method according to statement 15, wherein:
the configuration of chassis (105) is drawn from a set including a Non-Volatile Memory Express (NVMe) chassis (105) and a Non-Volatile Memory Express Over Fabric (NVMeoF) chassis (105); and
selecting (1239, 1254) a driver (815, 820) responsive to the configuration of the chassis (105) includes selecting (1239, 1254) one of an NVMe driver (815) and an NVMeoF driver (820) for the BMC (125) according to the configuration of the chassis (105).
Statement 17. An embodiment of the inventive concept includes a method according to statement 16, wherein:
determining (1203) by a Baseboard Management Controller (BMC) (125) a configuration of a chassis (105) including the BMC (125) includes determining (1236) by the BMC (125) that the configuration of the chassis (105) is the NVMeoF chassis (105); and
the method further comprises determining (1248), by the BMC (125), the configuration of at least one device (320, 325, 330) in the chassis (105) including the BMC (125).
Statement 18. An embodiment of the inventive concept includes a method according to statement 16, wherein determining (1203) by a Baseboard Management Controller (BMC) (125) a configuration of a chassis (105) including the BMC (125) includes reading (1305) the configuration of the chassis (105) from a Vital Product Data (VPD) (355).
Statement 19. An embodiment of the inventive concept includes a method according to statement 18, wherein the VPD (355) is stored in an Electrically Erasable Programmable Read Only Memory (EEPROM) (345).
Statement 20. An embodiment of the inventive concept includes a method according to statement 16, wherein determining (1203) by a Baseboard Management Controller (BMC) (125) a configuration of a chassis (105) including the BMC (125) includes accessing (1310) a signal from at least one pin (605, 610) on the BMC (125) to determine the configuration of the chassis (105).
Statement 21. An embodiment of the inventive concept includes a method according to statement 16, further comprising downloading (1242) the selected driver (815, 820) from a driver source (345, 910, 915).
Statement 22. An embodiment of the inventive concept includes a method according to statement 21, wherein the driver source (345, 910, 915) is drawn from a set including storage in an EEPROM (345), a first site (910) on a local computer network (905), and a second site (915) on a global computer network (905).
Statement 23. An embodiment of the inventive concept includes a method according to statement 16, wherein determining (1203) by a Baseboard Management Controller (BMC) (125) a configuration of a chassis (105) includes determining by the BMC (125) that the configuration of the chassis (105) is a High Availability (HA) chassis (105).
Statement 24. An embodiment of the inventive concept includes a method according to statement 23, wherein selecting a driver for the BMC (125) according to the configuration of the chassis (105) includes selecting (1209) an HA driver (825).
Statement 25. An embodiment of the inventive concept includes a method according to statement 24, further comprising reporting (1215) an error if the HA driver (825) is not available.
Statement 26. An embodiment of the inventive concept includes a method according to statement 24, further comprising attempting (1224) to communicate with a pairing partner (710) for the BMC (125).
Statement 27. An embodiment of the inventive concept includes a method according to statement 26, further comprising reporting (1227) an error if the BMC (125) may not communicate with the pairing partner (710).
Statement 28. An embodiment of the inventive concept includes a method according to statement 26, wherein attempting (1224) to communicate with a pairing partner (710) for the BMC (125) includes attempting (1224) to communicate with the pairing partner (710) for the BMC (125) after loading the HA driver (825).
Statement 29. An embodiment of the inventive concept includes article, comprising a tangible storage medium, the tangible storage medium having stored thereon non-transitory instructions that, when executed by a machine, result in:
determining (1203) by a Baseboard Management Controller (BMC) (125) a configuration of a chassis (105) including the BMC (125);
selecting (1239, 1254) a driver (815, 820) responsive to the configuration of the chassis (105); and
loading (1245) the selected driver (815, 820),
wherein the BMC (125) may self-configure without using any BIOS, device drivers, or operating systems.
Statement 30. An embodiment of the inventive concept includes an article according to statement 29, wherein:
the configuration of chassis (105) is drawn from a set including a Non-Volatile Memory Express (NVMe) chassis (105) and a Non-Volatile Memory Express Over Fabric (NVMeoF) chassis (105);
selecting (1239, 1254) a driver (815, 820) responsive to the configuration of the chassis (105) includes selecting (1239, 1254) one of an NVMe driver (815) and an NVMeoF driver (820) for the BMC (125) according to the configuration of the chassis (105).
Statement 31. An embodiment of the inventive concept includes an article according to statement 30, wherein:
determining (1203) by a Baseboard Management Controller (BMC) (125) a configuration of a chassis (105) including the BMC (125) includes determining (1236) by the BMC (125) that the configuration of the chassis (105) is the NVMeoF chassis (105); and
the tangible storage medium having stored thereon further non-transitory instructions that, when executed by the machine, result in determining (1248), by the BMC (125), the configuration of at least one device (320, 325, 330) in the chassis (105) including the BMC (125).
Statement 32. An embodiment of the inventive concept includes an article according to statement 30, wherein determining (1203) by a Baseboard Management Controller (BMC) (125) a configuration of a chassis (105) including the BMC (125) includes reading (1305) the configuration of the chassis (105) from a Vital Product Data (VPD) (355).
Statement 33. An embodiment of the inventive concept includes an article according to statement 32, wherein the VPD (355) is stored in an Electrically Erasable Programmable Read Only Memory (EEPROM) (345).
Statement 34. An embodiment of the inventive concept includes an article according to statement 30, wherein determining (1203) by a Baseboard Management Controller (BMC) (125) a configuration of a chassis (105) including the BMC (125) includes accessing (1310) a signal from at least one pin (605, 610) on the BMC (125) to determine the configuration of the chassis (105).
Statement 35. An embodiment of the inventive concept includes an article according to statement 30, the tangible storage medium having stored thereon further non-transitory instructions that, when executed by the machine, result in downloading (1242) the selected driver (815, 820) from a driver source (345, 910, 915).
Statement 36. An embodiment of the inventive concept includes an article according to statement 35, wherein the driver source (345, 910, 915) is drawn from a set including storage in an EEPROM (345), a first site (910) on a local computer network (905), and a second site (915) on a global computer network (905).
Statement 37. An embodiment of the inventive concept includes an article according to statement 30, wherein determining (1203) by a Baseboard Management Controller (BMC) (125) a configuration of a chassis (105) includes determining by the BMC (125) that the chassis (105) is a High Availability (HA) chassis (105).
Statement 38. An embodiment of the inventive concept includes an article according to statement 37, wherein selecting a driver for the BMC (125) according to the configuration of the chassis (105) includes selecting (1209) an HA driver (825).
Statement 39. An embodiment of the inventive concept includes an article according to statement 38, the tangible storage medium having stored thereon further non-transitory instructions that, when executed by the machine, result in reporting (1215) an error if the HA driver (825) is not available.
Statement 40. An embodiment of the inventive concept includes an article according to statement 38, the tangible storage medium having stored thereon further non-transitory instructions that, when executed by the machine, result in attempting (1224) to communicate with a pairing partner (710) for the BMC (125).
Statement 41. An embodiment of the inventive concept includes an article according to statement 40, the tangible storage medium having stored thereon further non-transitory instructions that, when executed by the machine, result in reporting (1227) an error if the BMC (125) may not communicate with the pairing partner (710).
Statement 42. An embodiment of the inventive concept includes an article according to statement 40, wherein attempting (1224) to communicate with a pairing partner (710) for the BMC (125) includes attempting (1224) to communicate with the pairing partner (710) for the BMC (125) after loading the HA driver (825).
Statement 43. An embodiment of the inventive concept includes a proxy device (125) in a chassis (105), comprising:
a device communication logic (430) to communicate with at least one device (320, 325, 330) over a control plane about data regarding the at least one device (320, 325, 330);
a reception logic (440) to receive a query (1105) from a host (110), the query (1105) requesting information about the at least one device (320, 325, 330); and
a transmission logic (445) to send a response (1110) to the host (110), the response (1110) including data about the at least one device (320, 325, 330).
Statement 44. An embodiment of the inventive concept includes a proxy device (125) according to statement 43, wherein the proxy device (125) is drawn from a set including a Baseboard Management Controller (BMC) (125), a Redundant Array of Independent Disks (RAID) controller, and a processor (305).
Statement 45. An embodiment of the inventive concept includes a proxy device (125) according to statement 43, wherein the at least one device (320, 325, 330) is drawn from a set including a storage device (320, 325, 330) and a Network Interface Card (NIC).
Statement 46. An embodiment of the inventive concept includes a proxy device (125) according to statement 43, wherein the device communication logic (430) includes a read logic (1005) to read the data regarding the at least one device (320, 325, 330) from a Vital Product Data (360) for the at least one device (320, 325, 330).
Statement 47. An embodiment of the inventive concept includes a proxy device (125) according to statement 46, wherein the Vital Product Data (360) is stored in an Electrically Erasable Programmable Read Only Memory (EEPROM) (345).
Statement 48. An embodiment of the inventive concept includes a proxy device (125) according to statement 43, wherein:
the device communication logic (430) includes a polling logic (1010) to poll the at least one device (320, 325, 330) for the data regarding the at least one device (320, 325, 330); and
the reception logic (440) is operative to receive the data regarding the at least one device (320, 325, 330) from the at least one device (320, 325, 330).
Statement 49. An embodiment of the inventive concept includes a proxy device (125) according to statement 43, wherein the chassis (105) includes permanent storage associated with the proxy device (125) in which the proxy device (125) may create a Log Page (1115) from the data regarding the at least one device (320, 325, 330).
Statement 50. An embodiment of the inventive concept includes a proxy device (125) according to statement 49, further comprising a Log Page creation logic (435) to create a Log Page (1115) from the data about the at least one device (320, 325, 330).
Statement 51. An embodiment of the inventive concept includes a proxy device (125) according to statement 49, wherein the transmission logic (445) is operative to send the Log Page (1115) to the host (110) responsive to the query (1105).
Statement 52. An embodiment of the inventive concept includes a method, comprising:
receiving (1405, 1410), at a proxy device (125), at least one data from at least one device (320, 325, 330) about configurations of the at least one device (320, 325, 330), the data from the at least one device (320, 325, 330) received over a control plane;
compiling (1415) the at least one data into a record (1115);
receiving (1420), at the proxy device (125), a query (1105) from a host (110) for the configurations of the at least one device (320, 325, 330); and
sending (1425) the record (1115) from the proxy device (125) to the host (110),
wherein the proxy device (125) may receive (1405, 1410) the at least one data from the at least one device (320, 325, 330) and compile (1415) the at least one data into a record (1115) before receiving (1420) the query (1105) from the host (110).
Statement 53. An embodiment of the inventive concept includes a method according to statement 52, wherein the proxy device (125) is drawn from a set including a Baseboard Management Controller (BMC) (125), a Redundant Array of Independent Disks (RAID) controller, a processor (305), or a software proxy device.
Statement 54. An embodiment of the inventive concept includes a method according to statement 52, wherein the at least one device (320, 325, 330) is drawn from a set including a storage device (320, 325, 330) and a Network Interface Card (NIC).
Statement 55. An embodiment of the inventive concept includes a method according to statement 52, wherein receiving (1405, 1410), at a proxy device (125), at least one data from at least one device (320, 325, 330) about configurations of the at least one device (320, 325, 330) includes receiving (1405, 1410), at the proxy device (125), the at least one data from the at least one device (320, 325, 330) about configurations of the at least one device (320, 325, 330) along a control plane.
Statement 56. An embodiment of the inventive concept includes a method according to statement 52, wherein receiving (1405, 1410), at a proxy device (125), at least one data from at least one device (320, 325, 330) about configurations of the at least one device (320, 325, 330) includes polling (1510) the at least one device (320, 325, 330) for the configurations of the at least one device (320, 325, 330).
Statement 57. An embodiment of the inventive concept includes a method according to statement 52, wherein receiving (1405, 1410), at a proxy device (125), at least one data from at least one device (320, 325, 330) about configurations of the at least one device (320, 325, 330) includes receiving (1405, 1410) a datum from one of the at least one device (320, 325, 330) when a configuration of the one of the at least one device (320, 325, 330) changes.
Statement 58. An embodiment of the inventive concept includes a method according to statement 52, wherein receiving (1405, 1410), at a proxy device (125), at least one data from at least one device (320, 325, 330) about configurations of the at least one device (320, 325, 330) includes reading (1505) the at least one data from at least one Vital Product Data (360).
Statement 59. An embodiment of the inventive concept includes a method according to statement 58, wherein the at least one Vital Product Data (360) are stored in an Electrically Erasable Programmable Read Only Memory (EEPROM) (345).
Statement 60. An embodiment of the inventive concept includes a method according to statement 52, wherein compiling (1415) the at least one data into a record (1115) includes creating (1610) a Log Page (1115) from the at least one data.
Statement 61. An embodiment of the inventive concept includes a method according to statement 60, wherein sending (1425) the configurations of the at least one device (320, 325, 330) from the proxy device (125) to the host (110) includes sending (1425) the Log Page (1115) from the proxy device (125) to the host (110).
Statement 62. An embodiment of the inventive concept includes a method according to statement 52, wherein sending (1425) the configurations of the at least one device (320, 325, 330) from the proxy device (125) to the host (110) includes sending (1425) the at least one data from the proxy device (125) to the host (110).
Statement 63. An embodiment of the inventive concept includes a method, comprising:
determining (1705), by a device (320, 325, 330), a change in a configuration of the device (320, 325, 330); and
notifying (1710, 1715, 1720, 1725) a proxy device (125) over a control plane about the change in the configuration of the device (320, 325, 330).
Statement 64. An embodiment of the inventive concept includes a method according to statement 63, wherein the proxy device (125) is drawn from a set including a Baseboard Management Controller (BMC) (125), a Redundant Array of Independent Disks (RAID) controller, a processor (305), or a software proxy device.
Statement 65. An embodiment of the inventive concept includes a method according to statement 63, wherein the at least one device (320, 325, 330) is drawn from a set including a storage device (320, 325, 330) and a Network Interface Card (NIC).
Statement 66. An embodiment of the inventive concept includes a method according to statement 63, wherein notifying (1710, 1715, 1720, 1725) a proxy device (125) about the change in the configuration of the device (320, 325, 330) includes writing (1710) the change in the configuration of the device (320, 325, 330) to a Vital Product Data (360) that may be read by the proxy device (125).
Statement 67. An embodiment of the inventive concept includes a method according to statement 66, wherein notifying (1710, 1715, 1720, 1725) a proxy device (125) about the change in the configuration of the device (320, 325, 330) further includes notifying (1715) the proxy device (125) that the change in the configuration of the device (320, 325, 330) was written to the Vital Product Data (360).
Statement 68. An embodiment of the inventive concept includes a method according to statement 63, wherein notifying (1710, 1715, 1720, 1725) a proxy device (125) about the change in the configuration of the device (320, 325, 330) includes:
receiving (1720) a query (1105) from the proxy device (125) about a current status of the configuration of the device (320, 325, 330); and
sending (1725) a response (1110) to the proxy device (125) including the change in the configuration of the device (320, 325, 330).
Statement 69. An embodiment of the inventive concept includes an embodiment of the inventive concept includes an article, comprising a tangible storage medium, the tangible storage medium having stored thereon non-transitory instructions that, when executed by a machine, result in:
receiving (1405, 1410), at a proxy device (125), at least one data from at least one device (320, 325, 330) about configurations of the at least one device (320, 325, 330), the data from the at least one device (320, 325, 330) received over a control plane;
compiling (1415) the at least one data into a record (1115);
receiving (1420), at the proxy device (125), a query (1105) from a host (110) for the configurations of the at least one device (320, 325, 330); and
sending (1425) the record (1115) from the proxy device (125) to the host (110),
wherein the proxy device (125) may receive (1405, 1410) the at least one data from the at least one device (320, 325, 330) and compile (1415) the at least one data into a record (1115) before receiving (1420) the query (1105) from the host (110).
Statement 70. An embodiment of the inventive concept includes an article according to statement 69, wherein the proxy device (125) is drawn from a set including a Baseboard Management Controller (BMC) (125), a Redundant Array of Independent Disks (RAID) controller, a processor (305), or a software proxy device.
Statement 71. An embodiment of the inventive concept includes an article according to statement 69, wherein the at least one device (320, 325, 330) is drawn from a set including a storage device (320, 325, 330) and a Network Interface Card (NIC).
Statement 72. An embodiment of the inventive concept includes an article according to statement 69, wherein receiving (1405, 1410), at a proxy device (125), at least one data from at least one device (320, 325, 330) about configurations of the at least one device (320, 325, 330) includes receiving (1405, 1410), at the proxy device (125), the at least one data from the at least one device (320, 325, 330) about configurations of the at least one device (320, 325, 330) along a control plane.
Statement 73. An embodiment of the inventive concept includes an article according to statement 69, wherein receiving (1405, 1410), at a proxy device (125), at least one data from at least one device (320, 325, 330) about configurations of the at least one device (320, 325, 330) includes polling (1510) the at least one device (320, 325, 330) for the configurations of the at least one device (320, 325, 330).
Statement 74. An embodiment of the inventive concept includes an article according to statement 69, wherein receiving (1405, 1410), at a proxy device (125), at least one data from at least one device (320, 325, 330) about configurations of the at least one device (320, 325, 330) includes receiving (1405, 1410) a datum from one of the at least one device (320, 325, 330) when a configuration of the one of the at least one device (320, 325, 330) changes.
Statement 75. An embodiment of the inventive concept includes an article according to statement 69, wherein receiving (1405, 1410), at a proxy device (125), at least one data from at least one device (320, 325, 330) about configurations of the at least one device (320, 325, 330) includes reading (1505) the at least one data from at least one Vital Product Data (360).
Statement 76. An embodiment of the inventive concept includes an article according to statement 75, wherein the at least one Vital Product Data (360) are stored in an Electrically Erasable Programmable Read Only Memory (EEPROM) (345).
Statement 77. An embodiment of the inventive concept includes an article according to statement 69, wherein compiling (1415) the at least one data into a record (1115) includes creating (1610) a Log Page (1115) from the at least one data.
Statement 78. An embodiment of the inventive concept includes an article according to statement 77, wherein sending (1425) the configurations of the at least one device (320, 325, 330) from the proxy device (125) to the host (110) includes sending (1425) the Log Page (1115) from the proxy device (125) to the host (110).
Statement 79. An embodiment of the inventive concept includes an article according to statement 69, wherein sending (1425) the configurations of the at least one device (320, 325, 330) from the proxy device (125) to the host (110) includes sending (1425) the at least one data from the proxy device (125) to the host (110).
Statement 80. An embodiment of the inventive concept includes an embodiment of the inventive concept includes an article, comprising a tangible storage medium, the tangible storage medium having stored thereon non-transitory instructions that, when executed by a machine, result in:
determining (1705), by a device (320, 325, 330), a change in a configuration of the device (320, 325, 330); and
notifying (1710, 1715, 1720, 1725) a proxy device (125) over a control plane about the change in the configuration of the device (320, 325, 330).
Statement 81. An embodiment of the inventive concept includes an article according to statement 80, wherein the proxy device (125) is drawn from a set including a Baseboard Management Controller (BMC) (125), a Redundant Array of Independent Disks (RAID) controller, a processor (305), or a software proxy device.
Statement 82. An embodiment of the inventive concept includes an article according to statement 80, wherein the at least one device (320, 325, 330) is drawn from a set including a storage device (320, 325, 330) and a Network Interface Card (NIC).
Statement 83. An embodiment of the inventive concept includes an article according to statement 80, wherein notifying (1710, 1715, 1720, 1725) a proxy device (125) about the change in the configuration of the device (320, 325, 330) includes writing (1710) the change in the configuration of the device (320, 325, 330) to a Vital Product Data (360) that may be read by the proxy device (125).
Statement 84. An embodiment of the inventive concept includes an article according to statement 83, wherein notifying (1710, 1715, 1720, 1725) a proxy device (125) about the change in the configuration of the device (320, 325, 330) further includes notifying (1715) the proxy device (125) that the change in the configuration of the device (320, 325, 330) was written to the Vital Product Data (360).
Statement 85. An embodiment of the inventive concept includes an article according to statement 80, wherein notifying (1710, 1715, 1720, 1725) a proxy device (125) about the change in the configuration of the device (320, 325, 330) includes:
receiving (1720) a query (1105) from the proxy device (125) about a current status of the configuration of the device (320, 325, 330); and
sending (1725) a response (1110) to the proxy device (125) including the change in the configuration of the device (320, 325, 330).
Consequently, in view of the wide variety of permutations to the embodiments described herein, this detailed description and accompanying material is intended to be illustrative only, and should not be taken as limiting the scope of the inventive concept. What is claimed as the inventive concept, therefore, is all such modifications as may come within the scope and spirit of the following claims and equivalents thereto.
This application claims the benefit of U.S. Provisional Patent Application Ser. No. 62/394,727, filed Sep. 14, 2016, which is incorporated by reference herein for all purposes. This application is related to U.S. patent application Ser. No. 15/256,495, filed Sep. 2, 2016, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/366,622, filed Jul. 26, 2016, both of which are incorporated by reference herein for all purposes. This application is related to U.S. patent application Ser. No. 15/345,507, filed, Nov. 7, 2016, which claims the benefit of U.S. Provisional Patent Application Ser. No. 62/394,726, filed Sep. 14, 2016, both of which are incorporated by reference herein for all purposes.
Number | Name | Date | Kind |
---|---|---|---|
6295567 | Bassman | Sep 2001 | B1 |
6345303 | Knauerhase et al. | Feb 2002 | B1 |
6427198 | Berglund et al. | Jul 2002 | B1 |
6611863 | Banginwar | Aug 2003 | B1 |
7120759 | Chiu et al. | Oct 2006 | B2 |
7143153 | Black et al. | Nov 2006 | B1 |
7512585 | Agarwal et al. | Mar 2009 | B2 |
7536486 | Sadovsky et al. | May 2009 | B2 |
7620854 | Kuttan et al. | Nov 2009 | B2 |
7873700 | Pawlowski et al. | Jan 2011 | B2 |
7882393 | Grimes et al. | Feb 2011 | B2 |
7944812 | Carlson et al. | May 2011 | B2 |
8065347 | DeMeyer et al. | Nov 2011 | B1 |
8180862 | Baker et al. | May 2012 | B2 |
8396981 | Lee et al. | Mar 2013 | B1 |
8667224 | Yu et al. | Mar 2014 | B1 |
8832327 | Lin | Sep 2014 | B1 |
8943234 | Voorhees et al. | Jan 2015 | B1 |
8949517 | Cohen et al. | Feb 2015 | B2 |
8953644 | Chandra et al. | Feb 2015 | B2 |
8998636 | Gomez et al. | Apr 2015 | B2 |
9047222 | Chandra et al. | Jun 2015 | B2 |
9092321 | Salessi | Jul 2015 | B2 |
9244877 | Yang et al. | Jan 2016 | B2 |
9253275 | Bhogal et al. | Feb 2016 | B2 |
9280357 | Shaver et al. | Mar 2016 | B2 |
9280504 | Ben-Michael et al. | Mar 2016 | B2 |
9389805 | Cohen et al. | Jul 2016 | B2 |
9400749 | Kuzmin et al. | Jul 2016 | B1 |
9460042 | Iskandar et al. | Oct 2016 | B2 |
9465756 | Bennett | Oct 2016 | B2 |
9648148 | Rimmer et al. | May 2017 | B2 |
9653124 | Heyd et al. | May 2017 | B2 |
9734093 | Khemani et al. | Aug 2017 | B2 |
9734106 | Kotzur et al. | Aug 2017 | B2 |
9785346 | Yost | Oct 2017 | B2 |
9785355 | Huang | Oct 2017 | B2 |
9785356 | Huang | Oct 2017 | B2 |
9811481 | Bhatia et al. | Nov 2017 | B2 |
9830082 | Srinivasan et al. | Nov 2017 | B1 |
9904330 | Schuette et al. | Feb 2018 | B2 |
9906596 | Sikdar | Feb 2018 | B2 |
9934173 | Sakalley | Apr 2018 | B1 |
9934183 | Brassac et al. | Apr 2018 | B2 |
9959240 | Mundt | May 2018 | B2 |
9965367 | Shih | May 2018 | B2 |
9990313 | Monji et al. | Jun 2018 | B2 |
10019388 | Long et al. | Jul 2018 | B2 |
10063638 | Huang | Aug 2018 | B2 |
10108450 | Pinto et al. | Oct 2018 | B2 |
10114778 | Worley et al. | Oct 2018 | B2 |
10162784 | Bassett et al. | Dec 2018 | B2 |
10206297 | Breakstone et al. | Feb 2019 | B2 |
10223313 | Shih | Mar 2019 | B2 |
10223316 | Mataya | Mar 2019 | B2 |
10235313 | Lee et al. | Mar 2019 | B2 |
10255215 | Breakstone et al. | Apr 2019 | B2 |
10275356 | Chou et al. | Apr 2019 | B2 |
10289517 | Beerens | May 2019 | B2 |
10289588 | Chu et al. | May 2019 | B2 |
10318443 | Su | Jun 2019 | B2 |
10346041 | Olarig et al. | Jul 2019 | B2 |
10372648 | Qiu | Aug 2019 | B2 |
10372659 | Olarig | Aug 2019 | B2 |
10394723 | Yang et al. | Aug 2019 | B2 |
10452576 | Stuhlsatz | Oct 2019 | B2 |
10467163 | Malwankar et al. | Nov 2019 | B1 |
10467170 | McKnight | Nov 2019 | B2 |
10560550 | Xue | Feb 2020 | B1 |
10866911 | Qiu et al. | Dec 2020 | B2 |
10901927 | Fischer et al. | Jan 2021 | B2 |
10929327 | Schrempp et al. | Feb 2021 | B1 |
10942666 | Pydipaty et al. | Mar 2021 | B2 |
11113046 | Bowen et al. | Sep 2021 | B1 |
11126352 | Olarig et al. | Sep 2021 | B2 |
20020087887 | Busam | Jul 2002 | A1 |
20020095491 | Edmonds et al. | Jul 2002 | A1 |
20020123365 | Thorson | Sep 2002 | A1 |
20040073912 | Meza | Apr 2004 | A1 |
20040111590 | Klein, Jr. | Jun 2004 | A1 |
20040147281 | Holcombe | Jul 2004 | A1 |
20040153844 | Ghose et al. | Aug 2004 | A1 |
20050025125 | Kwan | Feb 2005 | A1 |
20050060442 | Beverly et al. | Mar 2005 | A1 |
20050120157 | Chen et al. | Jun 2005 | A1 |
20060059287 | Rivard et al. | Mar 2006 | A1 |
20060095625 | Wootten et al. | May 2006 | A1 |
20060136621 | Tung et al. | Jun 2006 | A1 |
20080003845 | Hong | Jan 2008 | A1 |
20080288708 | Hsueh | Nov 2008 | A1 |
20090073896 | Gillingham | Mar 2009 | A1 |
20090077478 | Gillingham | Mar 2009 | A1 |
20090217188 | Alexander et al. | Aug 2009 | A1 |
20100077067 | Strole | Mar 2010 | A1 |
20100100858 | Schipper | Apr 2010 | A1 |
20100106836 | Schreyer et al. | Apr 2010 | A1 |
20100169512 | Matton et al. | Jul 2010 | A1 |
20110131380 | Rallens et al. | Jun 2011 | A1 |
20110151858 | Lai | Jun 2011 | A1 |
20120056728 | Erdmann et al. | Mar 2012 | A1 |
20120102580 | Bealkowski | Apr 2012 | A1 |
20120207156 | Srinivasan | Aug 2012 | A1 |
20120311654 | Dougherty, III et al. | Dec 2012 | A1 |
20130117503 | Nellans et al. | May 2013 | A1 |
20130117766 | Bax | May 2013 | A1 |
20130198311 | Tamir et al. | Aug 2013 | A1 |
20130242991 | Basso et al. | Sep 2013 | A1 |
20130282953 | Orme et al. | Oct 2013 | A1 |
20130304979 | Zimmer et al. | Nov 2013 | A1 |
20130311795 | Cong | Nov 2013 | A1 |
20130325998 | Hormuth et al. | Dec 2013 | A1 |
20140032641 | Du | Jan 2014 | A1 |
20140052928 | Shimoi | Feb 2014 | A1 |
20140122746 | Shaver | May 2014 | A1 |
20140195711 | Bhatia | Jul 2014 | A1 |
20140281458 | Ravimohan et al. | Sep 2014 | A1 |
20140330995 | Levy et al. | Nov 2014 | A1 |
20140344431 | Hsu et al. | Nov 2014 | A1 |
20150006758 | Holtman et al. | Jan 2015 | A1 |
20150039815 | Klein | Feb 2015 | A1 |
20150067188 | Chakhaiyar | Mar 2015 | A1 |
20150106660 | Chumbalkar et al. | Apr 2015 | A1 |
20150120874 | Kim et al. | Apr 2015 | A1 |
20150120971 | Bae et al. | Apr 2015 | A1 |
20150138900 | Choi | May 2015 | A1 |
20150178095 | Balakrishnan et al. | Jun 2015 | A1 |
20150181760 | Stephens | Jun 2015 | A1 |
20150205541 | Nishtala et al. | Jul 2015 | A1 |
20150234815 | Slik | Aug 2015 | A1 |
20150254088 | Chou et al. | Sep 2015 | A1 |
20150261434 | Kagan et al. | Sep 2015 | A1 |
20150301757 | Iwata et al. | Oct 2015 | A1 |
20150324312 | Jacobson et al. | Nov 2015 | A1 |
20150331473 | Jreji | Nov 2015 | A1 |
20150350096 | Dinc et al. | Dec 2015 | A1 |
20150370665 | Cannata et al. | Dec 2015 | A1 |
20150376840 | Shih | Dec 2015 | A1 |
20150381734 | Ebihara et al. | Dec 2015 | A1 |
20160004879 | Fisher et al. | Jan 2016 | A1 |
20160062936 | Brassac | Mar 2016 | A1 |
20160085718 | Huang | Mar 2016 | A1 |
20160092390 | Grothen et al. | Mar 2016 | A1 |
20160094619 | Khan et al. | Mar 2016 | A1 |
20160127468 | Malwankar et al. | May 2016 | A1 |
20160127492 | Malwankar et al. | May 2016 | A1 |
20160146754 | Prasad et al. | May 2016 | A1 |
20160188313 | Dubal et al. | Jun 2016 | A1 |
20160246754 | Rao et al. | Aug 2016 | A1 |
20160259597 | Worley et al. | Sep 2016 | A1 |
20160283428 | Guddeti | Sep 2016 | A1 |
20160306723 | Lu | Oct 2016 | A1 |
20160306768 | Mataya | Oct 2016 | A1 |
20160328344 | Jose et al. | Nov 2016 | A1 |
20160328347 | Worley et al. | Nov 2016 | A1 |
20160337272 | Berman | Nov 2016 | A1 |
20160366071 | Chandran et al. | Dec 2016 | A1 |
20170068268 | Giriyappa et al. | Mar 2017 | A1 |
20170068628 | Calciu et al. | Mar 2017 | A1 |
20170068630 | Iskandar et al. | Mar 2017 | A1 |
20170168943 | Chou et al. | Jun 2017 | A1 |
20170185554 | Fricker | Jun 2017 | A1 |
20170206034 | Fetik | Jul 2017 | A1 |
20170262029 | Nelson et al. | Sep 2017 | A1 |
20170269871 | Khan et al. | Sep 2017 | A1 |
20170270001 | Suryanarayana | Sep 2017 | A1 |
20170286305 | Kalwitz | Oct 2017 | A1 |
20170317901 | Agrawal et al. | Nov 2017 | A1 |
20170344259 | Freyensee et al. | Nov 2017 | A1 |
20170344294 | Mishra et al. | Nov 2017 | A1 |
20170357515 | Bower, III | Dec 2017 | A1 |
20180004695 | Chu et al. | Jan 2018 | A1 |
20180019896 | Paquet et al. | Jan 2018 | A1 |
20180032463 | Olarig | Feb 2018 | A1 |
20180032471 | Olarig | Feb 2018 | A1 |
20180074717 | Olarig et al. | Mar 2018 | A1 |
20180101492 | Cho et al. | Apr 2018 | A1 |
20180131633 | Li | May 2018 | A1 |
20180173652 | Olarig et al. | Jun 2018 | A1 |
20180210517 | Yun | Jul 2018 | A1 |
20180267925 | Rees | Sep 2018 | A1 |
20180275919 | Chirumamilla et al. | Sep 2018 | A1 |
20180373609 | Beerens | Dec 2018 | A1 |
20190104632 | Nelson et al. | Apr 2019 | A1 |
20190339888 | Sasidharan et al. | Nov 2019 | A1 |
20200293916 | Li | Sep 2020 | A1 |
20220188002 | Olarig | Jun 2022 | A1 |
Number | Date | Country |
---|---|---|
1641568 | Jul 2005 | CN |
101847429 | May 2012 | CN |
104025063 | Sep 2014 | CN |
104202197 | Dec 2014 | CN |
104572516 | Apr 2015 | CN |
104615577 | May 2015 | CN |
105260275 | Jan 2016 | CN |
105912275 | Aug 2016 | CN |
103412769 | Nov 2017 | CN |
2290497 | Mar 2011 | EP |
2843557 | Mar 2015 | EP |
2001290752 | Oct 2001 | JP |
4257050 | Apr 2009 | JP |
2010146525 | Jul 2010 | JP |
2011048534 | Mar 2011 | JP |
2012506184 | Mar 2012 | JP |
2013041390 | Feb 2013 | JP |
2014241545 | Dec 2014 | JP |
2015049742 | Mar 2015 | JP |
2015191649 | Nov 2015 | JP |
2015194005 | Nov 2015 | JP |
2015532985 | Nov 2015 | JP |
2016037501 | Mar 2016 | JP |
2016045968 | Apr 2016 | JP |
WO2015194005 | Apr 2017 | JP |
20090106469 | Oct 2009 | KR |
20120135205 | Dec 2012 | KR |
20150047785 | May 2015 | KR |
20150071898 | Jun 2015 | KR |
20160074659 | Jun 2016 | KR |
201445325 | Dec 2014 | TW |
2014209764 | Dec 2014 | WO |
2015049742 | Apr 2015 | WO |
2015191649 | Dec 2015 | WO |
2016037501 | Mar 2016 | WO |
2016085016 | Jun 2016 | WO |
Entry |
---|
Non-Final Office Action for U.S. Appl. No. 15/256,495, dated Mar. 29, 2018. |
Fang, Chin, “Using NVMe Gen3 PCle SSD Cards in High-density Servers for High-performance Big Data Transfer Over Multiple Network Channels”, SLAC National Accelerator Laboratory, Stanford University, Stanford, California, Feb. 7, 2015, 17 pages. |
Final Office Action for U.S. Appl. No. 15/256,495, dated Oct. 19, 2018. |
Office Action for U.S. Appl. No. 15/411,962, dated Aug. 10, 2018. |
Office Action for U.S. Appl. No. 15/256,495, dated Jun. 14, 2019. |
Advisory Action for U.S. Appl. No. 15/256,495, dated Feb. 1, 2019. |
Notice of Allowance for U.S. Appl. No. 15/345,507, dated Feb. 19, 2019. |
Notice of Allowance for U.S. Appl. No. 15/411,962, dated Mar. 18, 2019. |
Final Office Action for U.S. Appl. No. 15/411,962, dated Dec. 20, 2018. |
NVM Express over Fabrics specification Revision 1.0; NVM Express Inc.; Jun. 5, 2016. (Year: 2016). |
Office Action for U.S. Appl. No. 15/345,507, dated Dec. 3, 2018. |
Office Action for U.S. Appl. No. 16/202,079, dated Aug. 22, 2019. |
Office Action for U.S. Appl. No. 16/424,474, dated Oct. 15, 2019. |
Final Office Action for U.S. Appl. No. 15/256,495, dated Dec. 4, 2019. |
Office Action for U.S. Appl. No. 16/421,458, dated Dec. 30, 2019. |
Office Action for U.S. Appl. No. 16/424,474, dated Feb. 3, 2020. |
Notice of Allowance for U.S. Appl. No. 15/256,495, dated Mar. 5, 2020. |
Office Action for U.S. Appl. No. 16/202,079, dated Mar. 4, 2020. |
Corrected Notice of Allowability for U.S. Appl. No. 16/202,079, dated Jul. 22, 2020. |
Notice of Allowance for U.S. Appl. No. 16/424,474, dated Jul. 15, 2020. |
Office Action for U.S. Appl. No. 16/844,995, dated Sep. 4, 2020. |
Final Office Action for U.S. Appl. No. 16/424,474, dated May 1, 2020. |
Notice of Allowance for U.S. Appl. No. 16/202,079, dated Jun. 1, 2020. |
Notice of Allowance for U.S. Appl. No. 16/421,458, dated Apr. 15, 2020. |
Corrected Notice of Allowability for U.S. Appl. No. 15/256,495, dated Mar. 18, 2021. |
Corrected Notice of Allowability for U.S. Appl. No. 16/424,474, dated Feb. 22, 2021. |
Notice of Allowance for U.S. Appl. No. 16/202,079, dated Jan. 27, 2021. |
Notice of Allowance for U.S. Appl. No. 16/921,923, dated Feb. 18, 2021. |
Corrected Notice of Allowability for U.S. Appl. No. 16/424,474, dated Mar. 29, 2021. |
Corrected Notice of Allowability for U.S. Appl. No. 16/921,923, dated Apr. 1, 2021. |
Final Office Action for U.S. Appl. No. 16/844,995, dated Mar. 29, 2021. |
Notice of Allowance for U.S. Appl. No. 15/403,088, dated Oct. 22, 2018. |
OC3D, “What is the New U.2 SSD Connection?,” (https://www.overclock3d.net/news/storage/what_is_the_new_u_2_ssd_connection/1), Jul. 2016, retrieved Apr. 12, 2021, 5 pages. |
Office Action for U.S. Appl. No. 15/403,088, dated Jun. 7, 2018. |
Wikipedia, “SATA Express,” (https://en.wikipedia.org/wiki/SATA_Express), retrieved Apr. 12, 2021, 6 pages. |
Wikipedia, “U.2,” (https://en.wikipedia.org/wiki/U.2), retrieved Apr. 12, 2021, 2 pages. |
Office Action for U.S. Appl. No. 16/857,172, dated Oct. 8, 2020. |
Office Action for U.S. Appl. No. 16/921,923, dated Oct. 28, 2020. |
Corrected Notice of Allowability for U.S. Appl. No. 15/256,495, dated May 13, 2021. |
Corrected Notice of Allowability for U.S. Appl. No. 16/921,923, dated May 26, 2021. |
Notice of Allowance for U.S. Appl. No. 16/202,079, dated May 14, 2021. |
Notice of Allowance for U.S. Appl. No. 16/424,474, dated Apr. 30, 2021. |
Notice of Allowance for U.S. Appl. No. 16/857,172, dated May 3, 2021. |
Corrected Notice of Allowability for U.S. Appl. No. 16/202,079, dated Jul. 12, 2021. |
Corrected Notice of Allowability for U.S. Appl. No. 16/921,923, dated Jul. 14, 2021. |
Corrected Notice of Allowability for U.S. Appl. No. 16/202,079, dated Sep. 15, 2021. |
Corrected Notice of Allowability for U.S. Appl. No. 16/424,474, dated Aug. 18, 2021. |
Corrected Notice of Allowability for U.S. Appl. No. 16/857,172, dated Aug. 20, 2021. |
Corrected Notice of Allowability for U.S. Appl. No. 16/921,923, dated Aug. 24, 2021. |
Office Action for U.S. Appl. No. 17/099,776, dated Sep. 24, 2021. |
Notice of Allowance for U.S. Appl. No. 16/844,995, dated Sep. 29, 2021. |
Office Action for U.S. Appl. No. 17/022,075, dated Oct. 15, 2021. |
Corrected Notice of Allowability for U.S. Appl. No. 16/692,997, dated Jun. 18, 2020. |
Corrected Notice of Allowability for U.S. Appl. No. 16/844,995, dated Dec. 9, 2021. |
Corrected Notice of Allowability for U.S. Appl. No. 16/950,624, dated Jul. 16, 2021. |
Corrected Notice of Allowability for U.S. Appl. No. 16/950,624, dated Jun. 10, 2021. |
Final Office Action for U.S. Appl. No. 16/211,923, dated Aug. 19, 2019. |
Final Office Action for U.S. Appl. No. 16/692,997, dated Mar. 26, 2020. |
Final Office Action for U.S. Appl. No. 17/063,501, dated Nov. 2, 2021. |
Final Office Action for U.S. Appl. No. 17/063,507, dated Nov. 22, 2021. |
Notice of Allowance for U.S. Appl. No. 16/211,923, dated Sep. 13, 2019. |
Notice of Allowance for U.S. Appl. No. 16/692,997, dated Jun. 1, 2020. |
Notice of Allowance for U.S. Appl. No. 16/950,624, dated May 10, 2021. |
OC3D, “What is the New U.2 SSD Connection?” OC3D News, Jul. 2016 (https://www.overclock3d.net/news/storage/what_is_the_new_u_2_ssd_connection/1), retrieved May 2018, 4 pages. |
Office Action for U.S. Appl. No. 16/202,079, dated Dec. 9, 2021. |
Office Action for U.S. Appl. No. 16/211,923, dated May 6, 2019. |
Office Action for U.S. Appl. No. 16/692,997, dated Dec. 19, 2019. |
Office Action for U.S. Appl. No. 16/950,624, dated Jan. 25, 2021. |
Office Action for U.S. Appl. No. 17/063,501, dated Jul. 15, 2021. |
Office Action for U.S. Appl. No. 17/063,507, dated Aug. 6, 2021. |
SSD Form Factor Work Group, “Enterprise SSD Form Factor 1.0a”, 2021 SSD Form Factor Work Group, pp. 1-55. (Year: 2012). |
Wikipedia, “SATA Express”, (https://en.wwikipedia.org/wiki/SATA_Express), retrieved May 2018, 6 pages. |
Wikipedia, “U.2”, (https://en.wikipedia.org/wiki.U.2), retrieved May 2018, 3 pages. |
Notice of Allowance for U.S. Appl. No. 16/202,079, dated Mar. 17, 2022. |
Nortice of Allowance for U.S. Appl. No. 17/099,776, dated Feb. 15, 2022. |
Office Action for U.S. Appl. No. 16/844,995, dated Feb. 22, 2022. |
Office Action for U.S. Appl. No. 17/063,501, dated Feb. 24, 2022. |
Office Action for U.S. Appl. No. 17/063,507, dated Mar. 17, 2022. |
Final Office Action for U.S. Appl. No. 17/022,075, dated May 25, 2022. |
Notice of Allowance for U.S. Appl. No. 17/063,501, dated Jun. 2, 2022. |
Supplemental Notice of Allowability for U.S. Appl. No. 17/099,776, dated May 16, 2022. |
Corrected Notice of Allowability for U.S. Appl. No. 16/202,079, dated Jul. 7, 2022. |
Corrected Notice of Allowability for U.S. Appl. No. 17/063,501, dated Jul. 7, 2022. |
Corrected Notice of Allowability for U.S. Appl. No. 17/063,501, dated Jun. 23, 2022. |
Corrected Notice of Allowability for U.S. Appl. No. 17/099,776, dated Jun. 20, 2022. |
Final Office Action for U.S. Appl. No. 17/063,507, dated Jun. 27, 2022. |
Corrected Notice of Allowability for U.S. Appl. No. 16/202,079, dated Aug. 17, 2022. |
Corrected Notice of Allowability for U.S. Appl. No. 17/063,501, dated Jul. 27, 2022. |
Corrected Notice of Allowability for U.S. Appl. No. 17/099,776, dated Jul. 21, 2022. |
Office Action for U.S. Appl. No. 17/408,365, dated Aug. 2, 2022. |
Number | Date | Country | |
---|---|---|---|
20180074984 A1 | Mar 2018 | US |
Number | Date | Country | |
---|---|---|---|
62394727 | Sep 2016 | US |