Patent Grant
10564688
Embodiments described herein provide methods and systems for implementing resource management for an infrastructure (e.g., a datacenter infrastructure). Resource management includes datacenter byproduct management interfaces, datacenter power management, datacenter operations optimization and infrastructure resource management. At a high level, resource management facilitates using physical resources, including incidental physical resources, that are generated during operation of an infrastructure based on a minimum threshold reserve of the physical resource associated with the operating the infrastructure. Resource management can include controlling an amount of the physical resource that is generated and an amount the physical resource that is reserved. The minimum threshold reserve in combination with the control over generating and reserving the physical resource help identify an allocable amount of the physical resource.
A resource management system facilitates defining and allocating resources associated with the infrastructure. In particular, the resource management system processes resources and requests for resources based on a resource management framework built on Application Programming Interfaces (API) and resource management components. Physical resources of an infrastructure can be quantified to support resource management. Quantifying physical resources can be associated with devices of the infrastructure and/or requests for physical resources in the infrastructure to perform operations. Requests for physical resources of the infrastructure can be received from external entities that need the physical resources.
In one embodiment, an instance (e.g., device instance) is a computed representation of an infrastructure device in relation to a physical resource (e.g., power, water, carbon dioxide and heat) and a reserve of the physical resource, where the instance supports determining allocable units of the physical resource. In another embodiment, an instance (e.g., operation instance) represents an amount of a physical resource needed to perform an anticipated operation; the instance is also used to determine allocable units of the physical resource. The resource management system can monitor physical resources and devices using device instances and allocate the physical resources based on operation instances. As such, the resource management system can communicate an amount of allocated units for a request of a physical resource based on an amount of units of the physical resource that is being generated and an amount of the physical resource that is reserved.
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used in isolation as an aid in determining the scope of the claimed subject matter.
Embodiments are described in detail below with reference to the attached drawing figures, wherein:
Datacenters can provide housing for computing systems and associated components. The computing systems and components can support compute and storage operations for a variety of compute workloads and operations. Datacenters also include includes several devices or apparatus that consume and produce physical resources (e.g., power, water, heat and carbon dioxide). For example, large datacenters can use a significant amount of electrical power and water and produce heat and carbon dioxide.
A growing number of industries are utilizing datacenter resources, which has led to an ever-increasing amount of resource usage at datacenters and consequently the production of byproducts that can negatively impact the environment. As such, a comprehensive approach for operating datacenters for managing the increased use of resources, in particular physical resources, and controlling the increased production of byproducts can advantageously improve the operation of datacenters and reduce the negative consequences on the environment.
Embodiments described herein provide simple and efficient methods and systems for implementing resource management (e.g., datacenter byproduct management interfaces, datacenter power management, datacenter operations optimization, and infrastructure resource management) for infrastructures. At a high level, the resource management system provides a system for datacenter providers to operate datacenters with optimum efficiency, where efficient use of resources, especially physical resources, in a datacenter leads to achieving maximum productivity with minimum wasted resources. Also, the resource management system provides the capacity to leverage incidental and sometimes inevitable byproducts (i.e., physical resources as byproducts) of operating datacenter as resources in and around the datacenter. In some cases, byproducts may previously have been considered waste products of the datacenter. Nonetheless, the resource management system can efficiently generate, reserve, process, consume and distribute the byproduct physical resources of the datacenter. As such, the increased efficiency and usage of byproducts reduce the environmental impact of a datacenter operating based on the resource management system.
In operation, resource management facilitates using physical resources, including incidental physical resources, that are generated during operation of an infrastructure based on a minimum threshold reserve of the physical resource associated with the operating the infrastructure. Resource management can include controlling an amount of the physical resource that is generated and an amount the physical resource that is reserved. The minimum threshold reserve in combination with the control over generating and reserving the physical resource is used to identify an allocable amount of the physical resource.
The resource management system facilitates defining and allocating resources associated with the infrastructure. In particular, the resource management system processes resources and requests for resources based on a resource management framework built on Application Programming Interfaces (API) and resource management components. Physical resources of an infrastructure can be quantified to support resource management. Quantifying physical resources can be associated with devices of the infrastructure and/or requests for physical resources in the infrastructure to perform operations. Requests for physical resources of the infrastructure can be received from external entities that need the physical resources.
In one embodiment, an instance (e.g., device instance) is a computed representation of an infrastructure device in relation to a physical resource (e.g., power, water, carbon dioxide, heat) and a reserve of the physical resource, where the instance supports determining allocable units of the physical resource. In another embodiment, an instance (e.g., operation instance) represents an amount of a physical resource needed to perform an anticipated operation; the instance is also used to determine allocable units of the physical resource. The resource management system can monitor physical resources and devices using device instances and allocate the physical resources based on operation instances.
As such, the resource management system can communicate an amount of allocated units for a request of a physical resource based on an amount of units of the physical resource that is being generated and an amount of the physical resource that is reserved. The resource management system can support different types of implementations including: a datacenter byproduct management interface system, a datacenter power management system, a datacenter operations optimization system, and an infrastructure resource management system, as discussed in more detail below.
Datacenter Byproduct Management System
By way of example, for illustration of a first embodiment described herein, a datacenter can support a datacenter byproduct management system. The datacenter includes several types of devices. The devices operate to perform functions that support operations in the datacenter. For example, servers operate to provide compute resources or storage resources and a fan operates produce a current of air for cooling the datacenter. With the datacenter byproduct management system, datacenter devices can further be described as byproduct devices. A byproduct device consumes or supplies a specific byproduct—i.e., a physical resource (e.g., power, water, carbon dioxide—CO2, heat) of the datacenter. The datacenter byproduct management system provides a comprehensive view of physical resources that are available at the datacenter. Physical resources can be generated intentionally or incidentally by the datacenter devices. In either case, the physical resource is tracked and can further be identified as an intentionally produced physical resource or an incidental physical resource. For example, an electrical device can produce electricity intentionally but generate heat and water incidentally. The datacenter byproduct management system operates to efficiently generate, reserve, use and distribute physical resources and in particular process and utilize byproduct physical resources that would other be regarded as waste. The datacenter byproduct management system can specifically rely on technology-based techniques for implementation of resource management.
The datacenter byproduct management system operates using an Application Programming Interface (API) that supports a protocol for performing operations based on byproduct resources. At a high level, the API allows the datacenter including datacenter devices and physical resources to be expressed in terms of operations, inputs, outputs, and underlying classification types for characteristics of the datacenter. The API can define functionalities that are independent of specific implementations, allowing definitions and implementations to vary without compromising interfaces. The API can support representing, processing and communicating information associated with the physical resources. Physical resources can be generated, reserved, allocated and communicated based the APIs of the datacenter byproduct management system. In particular, the API can define a byproduct device in relation to the physical resource and a reserve of the physical resource. Using the APIs and byproduct management system, a byproduct device can be identified and classified with one or more physical resources (e.g., power devices, water devices, CO2 devices or heat devices). The APIs define units for quantifying the physical resource including a generation capacity of units of the physical resource by the byproduct device and reserve units of the physical resource associated with the byproduct device.
The datacenter byproduct management system also operates based on minimum threshold limits of physical resources associated with operating at least the datacenter. The minimum threshold limit of a physical resource can refer to a benchmark attribute for guidance on how much of the physical resource is needed to operate the datacenter. A breach of the minimum threshold can be configured to trigger remediation operations. The minimum threshold limit can be defined based on datacenter processes and performance metrics for the individual and combinations of physical resources. The minimum threshold limit of the physical resource can also be calculated and represented based on units defined in the API. The minimum threshold limit, quantified using the API, indicates an amount of the physical resource needed to operate the datacenter.
In operation, a byproduct resource manager of the datacenter byproduct management system can receive data for a byproduct device associated with the datacenter. It is contemplated that the byproduct resource manager continuously and automatically monitors byproduct devices and generates and updates device instances to provide a comprehensive view of physical resources. The byproduct resource manager can then use the data to generate a representation (e.g., device instance) byproduct device. In particular, a generation capacity of units of the physical resource and reserve units of the physical resource are identified. The device instance can be used to determine allocable units of the physical resource. The byproduct resource manager can specifically communicate with the byproduct device on an amount of units of the physical resource to be generated and an amount resource to be reserved based on the device instance and the minimum threshold limit of the physical resource.
An amount of allocated units can be identified from the allocable units and communicated as available for use. The amount of allocated units can be generated based on a request having requested units of a physical resource. A remote resource interface can communicate the request and the byproduct resource manager can identify allocable units and determine allocated units for the request. In one embodiment, the request can be used to generate an operation instance for processing the request with the same API-based representation of the device instance.
Datacenter Power Management System
By way of example, for illustration of a second embodiment described herein, a power management system is provided for leveraging power devices in a datacenter to support electrical infrastructures (e.g., a power grid). A datacenter can include several power devices (e.g., servers, racks hardware, network devices, batteries, etc.) that consume or supply power available at the datacenter. In one unique example, a power device can be an uninterruptable power supply (UPS) or a power generator. At a high level, the UPS can operate to consume power when charging batteries in the UPS and supply power stored in the batteries. The power generator in its own right operates to generate power.
The resource management APIs can provide a representation of power devices in the power management system. The resource management APIs can be configured to support representing electrical attributes associated with power devices and using the API representations (e.g., device instances, operation instances) that include the power device data to perform resource management. Sensors associated with power devices can help track electrical attributes represented using the APIs. Electrical attributes can refer to measureable electrical quantities that are parameters and operational characteristics of the electrical devices. Electrical attributes can include leading factor, lagging factor, reactive power, active power factor correction, power factor penalty, frequency firming, voltage correction, inductive loads, capacitive loads, phase correction and so on. In this regard, power device can be further associated with a power management scheme for electrical attributes of the power device. The power management scheme supports generating and reserving power based on corresponding electrical devices and electricity processing techniques associated with the power device. Using the APIs, the UPS, the generator and other power devices can be identified and classified as a power devices associated with power as a resource of the datacenter.
The datacenter power devices and electrical attributes can be used to shape power in that the power devices can be used to control generation, reservation, consumption, allocation and distribution of datacenter power. For example, the power device data can be used to change the datacenter from a lagging power factor to a leading power factor. Reactive power can be preserved and used as real power when needed. In particular, the scale of power usage and power devices at massive datacenters can significantly affect power management when performed based on the plurality of power devices in the datacenter, allowing the fractional changes in individual power devices to in the aggregate translate to significant results in shaping power. Shaping power, advantageously, yields lower charges from the utility companies. The power management systems can operate metaphorically as a shock absorber for a corresponding power grid.
By way of background, a power grid can experience micro-deficits in power (e.g., in the middle of the day there is more solar than in middle of night) and the fluctuation in supply and demand of power provided by a power utility provider impacts the power grid efficiency and functionality. The power management system can mitigate the impact of micro-deficits on the grid and help counter the inefficiencies of a power grid. In one example, the capacity of the datacenter to generate power and store power can be leveraged to compensate for any power drawn away from the datacenter. A minimum threshold limit of the power for the datacenter can also be calculated based on units defined in the API. The minimum threshold limit indicates an amount of power needed to operate the datacenter. The minimum threshold limit of power can be based on a plurality of different internal and external factors. For example, an assessment of the performance of the datacenter based on workloads, operating environment, operations resources and physical resources can be used as a benchmark for setting minimum threshold limits for physical resources at the datacenter. A minimum threshold manager can be a portion of the resource management system that provides the threshold limits to be applied to resource management. It is contemplated that the APIs provide support for representing the minimum threshold data for performing evaluations with other representations in the resource management system. As, such, advantageously, the datacenter operates to allow for other renewable sources of energy to be part of the power grid in that the power management systems compensates for the micro-deficits that are common when using renewable sources of power. Further, the datacenter operates as a peaking plant with less environment impact than traditional peaking plants that emit greenhouse gases.
In operation, power device data can be received at a power manager and represented as a device instance. For example, the UPS can be represented as a device instance having a plurality of attributes—classification, generation capacity units of power, and reserved units of power. The device instance along with the minimum threshold reserve of power associated with operating the datacenter can be used to determine allocable units of power. The power manager can also specifically instruct power devices as to an amount of units of power to be generated and an amount of units of power to be reserved. Generating power can be based on the power device (e.g., generator) producing power in part by transmitting power from a generation source to be made available for consumption via the datacenter or the power device (e.g., fans, computing devices, power supplies) throttling usage of power such that an unused portion of power is made available for consumption via the datacenter. An amount of allocated units of power, defined based on the amount of power that is being generated and the amount of units of power that is reserved can be provided for a request and communicated to a power requesting interface.
Datacenter Operations Optimization System
By way of example, for illustration of a third embodiment described herein, a datacenter optimization system is provided for a byproduct management system for a datacenter that operates in combination with a co-located operating infrastructure. A co-located infrastructure can refer to one of the following: a power grid, an agricultural facility, a manufacturing facility, a building, and a remote datacenter. As discussed above, the datacenter includes several types of devices (byproduct devices) that operate to consume or supply a specific byproduct—i.e., a physical resource (e.g., power, water, carbon dioxide—CO2, heat) of the datacenter. The co-located operating infrastructure can similarly be represented using the byproduct management system. Using the APIs and byproduct management system, a byproduct device, in the datacenter or the co-located operating infrastructure can be identified and classified with one or more physical resources (e.g., power devices, water devices, CO2 devices or heat devices). The APIs define units for quantifying the physical resource including a generation capacity of units of the physical resource by the byproduct device and reserve units of the physical resource associated with the byproduct device. A minimum threshold limit of the physical resource can also be calculated based on units defined in the API. The minimum threshold limit indicates an amount of the physical resource needed to operate both the datacenter and the co-located operating infrastructure. The minimum threshold limit of power can be based on a plurality of different internal and external factors of the datacenter and the co-located operating infrastructure. For example, an assessment of the performance of the datacenter and the co-located infrastructure based on workloads, operating environment, operations resources and physical resources can be used as a benchmark for setting minimum threshold limits for physical resources at the datacenter and the co-located infrastructure. A minimum threshold manager can be a portion of the datacenter operations optimization system that provides the threshold limits to be applied to resource management. In this regard, the datacenter generation capacity, reserves, and minimum thresholds can be based on the combination of the datacenter and the co-located operating infrastructure.
In operation, a byproduct resource manager can receive data for a byproduct device associated with the datacenter. The byproduct resource manager can then use the data to generate a representation (e.g., instance) of the byproduct device. In particular, a generation capacity of units of the physical resource and reserve units of the physical resource are identified. The device instance can be used to determine allocable units of the physical resource. An amount of allocated units can be identified from the allocable units and communicated as available for use.
In one embodiment, the remote resource interface is associated with a second datacenter. The datacenter operations optimization system can provide a compute resource manager that has access to the availability of compute resources at the first datacenter. In particular, the availability of compute resource can be due to availability of physical resources. For example, a generator at the first datacenter may be undergoing maintenance process that requires the generator to run for a period of time. The compute resource manager of the datacenter operations optimizing system can identify the maintenance process as a resource management opportunity because of the additional compute capacity available at the datacenter, at least for a period when the generator is undergoing the maintenance run. As such, a computer workload, based on a request for compute resources from the second datacenter, can be received and processed at the first datacenter on behalf of the second data center.
Further, the byproduct resource manager and a remote resource interface that receives the allocated units of the physical resource (or compute resources that are based on the availability of a physical resource), can also communicate corresponding notifications. The notifications can be communicated to start running one or more physical devices within the datacenter and another operating infrastructure that is associated with the remote resource interface. The notifications initiate operation processes for processing the amount of allocated units of the one or more physical resources. For example, the datacenter may turn on a grid connector to support power variability of the grid, while the power grid starts operating power components within the grid that will use the allocated units.
Infrastructure Management System
By way of example, for illustration of a fourth embodiment described herein, an infrastructure management system is provided. The infrastructure management system operates with the APIs defined above and also the framework of defining a minimum threshold of a physical resource needed to operate the infrastructure. In this regard, the infrastructure management system receives infrastructure device data and generates device instances for the devices to support resource management functionality. In addition, the infrastructure management system, in particular, defines a representation (e.g., an operation instance) of an anticipated operation of a device in the infrastructure. The operation instance includes requested physical resource units that indicate an amount of a physical resource that is needed in anticipation of an infrastructure device operation. For example, the infrastructure device is an electrical appliance (e.g., manufacturing machinery, refrigerator, etc.) that is associated with the infrastructure (e.g., manufacturing plant, building facility etc.) and one or more additional infrastructure devices in the infrastructure are computing devices within the infrastructure. The anticipated need of the electrical appliance and a minimum threshold reserve of power that is associated with operating the one or more additional infrastructure devices are used to determine an amount of units of power to be generated and an amount of units power to be reserved.
The device instances can be used to determine allocable units of the physical resource. An identified amount of allocable units and a determination of allocated units of the physical resource can be based on an amount of units of the physical resource that is being generated and an amount of units of the physical resource that is reserved. Operating instructions including the amount of allocated units of the physical resource for the operational instance are then communicated to the infrastructure device. For example, the amount of allocated units of the physical resource is communicated to the electrical appliance and instructions to begin the anticipated electrical appliance operation.
The resource management system, implemented as byproduct management interfaces, datacenter power management, datacenter operations optimization and infrastructure resource management provides a system for datacenter providers to operate datacenters with optimum efficiency, where efficient use of resources in a datacenter leads to achieving maximum productivity with minimum wasted resources. Also the resource management system provides that capacity to utilize byproduct physical resources and incidental and unintended byproduct physical resources of operating datacenter as resources in and around the datacenter.
Resource Management System
With reference to
With continued reference to
The resource management system includes the plurality of byproduct devices 102. A byproduct device can operate to perform operations in the datacenter 120A to support functionality of the datacenter. The byproduct device can also be associated with physical resources, referred to as byproduct device physical resources. As used herein, the byproduct can be associated with physical resources that are intentionally or incidentally generated. For example, byproduct device 120A is associated with physical resources 126A, 126B and 126C, byproduct device 120B is associated with physical resources 126A and 126B, and byproduct device 120C is associated with physical resources 126A and 126B and byproduct device 120D is associated with physical resource 120A, 126B, 126C and 126D. The byproduct devices operate to consumer or supply the associated physical resources. Byproduct devices can also operate to conserve physical resources. Byproduct devices can communicate byproduct device data (e.g., byproduct device data 122) to support functionality described herein. The byproduct device data can be generated based on sensors or other data tracking mechanisms associated with byproduct devices. The data that is measured and communicated at byproduct device can be constructed and communicated based on APIs of the resource management system 100.
The resource management system 100 includes the remote resource interface 130. The remote resource interface 130 can be associated with a remote resource operating environment 130A (e.g., a power grid, a greenhouse, or a manufacturing plant). The remote resource operating environment 130A is associated with the remote resource interface to request (e.g., requested units 32) units of a physical resource available at the datacenter. The remote resource interface can be configured to access resource requirements of the remote resource operating environment 130A and communicate requested units of a physical resource. The requested units 132 can be constructed and communicated based on APIs of the resource management system 100. Requested units of a physical resource can be received and then used to generate an operation instance. However, it is further contemplated that the requested unit may be constructed as operation instances at the remote resource interfaces, in the alternative. The remote resource interface 130 receives allocated units 134 of the physical resource that is requested. The remote resource interface can communicate notifications that further trigger components within the remote resource operating environment 130 to utilize the allocated units of the physical resource.
The byproduct resource manager 110 includes the API 114 that supports a protocol for performing operations based on byproduct resources. The API allows the datacenter including datacenter devices and physical resources to be expressed in terms of operations, inputs, outputs, and underlying classification types for characteristics of the datacenter. The API can define functionalities that are independent of specific implementations, allowing definitions and implementations to vary without compromising interfaces. The API can support representing, processing and communicating information associated with the physical resources.
The byproduct resource manager 110 also includes the byproduct threshold manager 116. The byproduct threshold manager 116 support the resource management system 100 to operate based on minimum threshold limits of physical resources. The minimum threshold limit of a physical resource can refer to a benchmark attribute for guidance on how much of the physical resource is needed to operate the datacenter. A breach of the minimum threshold can be configured to trigger remediation operations. For example, alarms can be triggered or the datacenter can selectively shut operation of non-essential devices to conserve the physical resources. Other variations and combination of remediation operations are contemplated with embodiments described herein. The minimum threshold limit can be defined based on datacenter processes and performance metrics for the individual and combinations of physical resources. For example, an assessment of the performance of the datacenter based on workloads, operating environment, operations resources and physical resources can be used as a benchmark for setting minimum threshold limits for physical resources at the datacenter. The byproduct threshold manager 116 provides the threshold limits to be applied to resource management. It is contemplated that the APIs provide support for representing the minimum threshold data for performing evaluations with other representations in the resource management system.
The byproduct resource manager 110 uses the resource management framework including the API 114 and the byproduct threshold manager 116 to quantify resources of the datacenter. The resources are physical resources that are associated with the datacenter. The byproduct resource manager 110 can quantify resources in two situations—for devices and for requests. Quantifying resources can include generating an instance. A device instance (e.g., device instance 140) can refer to a computed representation of a datacenter device in relation to a physical resource and a reserve resource of the resource. An operation instance (e.g., operation instance 150) can refer to an amount of a physical resource needed to perform an anticipated operation. Some implementations of resource management system 100 can operate optionally based on requested units from a remote resource interface that communicates the requested units for a physical resource. As discussed in more detail with reference to infrastructure management system in
The byproduct resource manager 110 controls an amount of the physical resource that is generated and an amount the physical resource that is reserved for a device from the plurality of devices. The byproduct resource manager 120 controls the device based on the device instance. For example, for a device 102A, device 102A can produce power. The device 102A can have a device instance that indicates physical resources associated with the devices. The byproduct resource manager 110 can communicate operation instructions 124 to the plurality of byproduct devices 120 to control generating and reserving physical resources associated with the byproduct devices. The byproduct resource manager 110 in particular controls the amount of a physical resource that is generated and the amount of physical resource that is reserved based on a minimum threshold resource. It is contemplated that byproduct resource manager may communicate control such that any value or no value for an amount of a physical resource to be generated and amount of physical resource to be reserved. Controls can be communicated dynamically based on changes in the infrastructure. The minimum threshold units of physical resources can be accessed via the byproduct threshold manager 116. The minimum threshold reserve in combination with the control over generating and reserving the physical resources help identify an allocable amount (e.g., allocable units 160) of the physical resource. An allocated amount (e.g., allocated units) of the allocable amount can be communicated to the remote resource interface 130. In operation, the resource management system can support different types of implementations including—a datacenter byproduct management interface system, a datacenter power management system, a datacenter operations optimization system, and an infrastructure resource management system.
Datacenter Byproduct Management System
With continued reference to
The byproduct resource manager 110 communicates operation instructions to the byproduct device for an amount of units of the physical resource to be generated and an amount of units of the physical resource to be reserved, based on the device instance and a minimum threshold reserve of the physical resource associated with operating the datacenter. The byproduct resource manager 110 automatically updates an amount of allocable units of the physical resource based on an amount of units of the physical resource that is being generated and an amount of units of the physical resource that is reserved.
The physical resource to be generated is generated based on the byproduct device producing the physical resource. As used herein, producing the physical resource can refer to transmitting the physical resource from a generation source to be made available for consumption via the datacenter. Producing the physical resource can also be based on the byproduct device throttling usage of the physical resource such that an unused portion of the physical resource is made available for consumption via the datacenter. Reserving the physical resource can be based on an internal component (e.g., UPS battery) or an external component (e.g., independent batter cells) associated with the byproduct device.
The remote resource interface 130 configured can generate a request having requested units of the physical resource. The byproduct resource manager can receive the request and generate an operation instance for processing the request. The remote resource interface can then receive an amount of allocated units for the request based on an amount of units of the physical resource that is being generated and an amount of units of the physical resource that is reserved.
It is contemplated that the byproduct resource manager and the remote resource interface can communicate corresponding notifications to actuate one or more physical devices within the datacenter and an operating infrastructure providing an operating environment for remote resource interface. The notification initiates operation processes for processing the amount of allocated units of the physical resource.
Turning now to
At block 230, a request having requested units of the physical resource is received. The request can be received from a remote resource interface. The request can be received based on APIs that are supported at the remote resource interface such that the request is provided in based on protocols of the API. Upon receiving the request, the byproduct resource manager can communicate operation instructions to the byproduct device for an amount of units of the physical resource to be generated and an amount of units of the physical resource to be reserved, based at least in part on the device instance and the minimum threshold reserve of the physical resource associated with the datacenter. An operation instance can be generated, the operation instance comprising requested units that indicate an amount of a physical resource needed in anticipation of an operation. The physical resource being generated is generated based on the byproduct device throttling usage of the physical resource such that an unused portion of the physical resource is made available for consumption via the datacenter. Reserving the physical resource is based on an internal component or an external component associated with the byproduct device. At block 240, an amount of allocated units for the request based on an amount of units of the physical resource that is being generated and an amount of units of the physical resource that are reserved, is communicated. The amount of units of the physical resources being generated and the amount of units of the physical resource being reserved is based at least in part on the device instance and a minimum threshold reserve of the physical resource associated with operating the datacenter.
Turning now to
At block 330, an amount of allocated units of the physical resource for the request based on an amount of units of the physical resource that is being generated and an amount of units of the physical resource that is reserved is received. The amount of units of the physical resources being generated and the amount of units of the physical resource being reserved is based at least in part on the device instance and the minimum threshold reserve of the physical resource associated with the datacenter. A notification to actuate one or more physical devices within an operating infrastructure that is the operating environment of the remote device is communicated to initiate operation processes for the processing the amount of allocated units of the physical resource. The operating infrastructure is selected from one of the following: a power grid, a co-located agricultural facility, a manufacturing facility, a building, and a remote datacenter. The byproduct resource manager upon receiving the request communicates operation instructions to the byproduct device for an amount of units of the physical resource to be generated and the amount of units of the physical resource to be reserved, based on the device instance and the minimum threshold reserve of the physical resource associated with operating the datacenter.
Datacenter Power Management System
With reference to
In operation, the power manager 410 receives power device data for a power device associated with the datacenter 420A. The use of datacenter is not meant to be limiting. Other types of infrastructure supporting a plurality of power devices that can be monitored as discussed herein are contemplated. The power manager 410 can generate based on the power device data 422, a device instance (e.g., power device instance 440) having a plurality of attributes associated with the power device, the device instance comprising a generation capacity of units of power and reserve units of power. The device instance is a computed representation of the power device in relation to power as a physical resource and a reserve of power, the instance supports determining allocable units of power. The power manager 410 communicates operation instructions (e.g., operation instructions 424) to the power device for an amount of units of power to be generated and an amount of units of power to be reserved, based on the device instance and a minimum threshold reserve of power associated with operating the datacenter.
The power manager 410 automatically updates an amount of allocable units of power based on an amount of units of power that is being generated and an amount of units of power that is reserved. The power to be generated is generated based on the power device producing power in part by transmitting power from a generation source to be made available for consumption via the datacenter or the power device throttling usage of power such that an unused portion of power is made available for consumption via the datacenter. The power to be reserved is reserved based an internal component or an external component associated with the power device.
The remote power interface 430 can generate a request (e.g., requested units 432) having requested units of power. The remote power interface can communicate the requested units 432 to receive allocated units (e.g., allocated units 434) of power from the power manager 410. The power manager can receive the request and generate an operation instance (e.g., operation instance 450) for processing the request. The remote power interface receives an amount of allocated units for the request based on an amount of units of power that is being generated and an amount of units of power that is reserved. In particular, the amount of units of power that is being generated and an amount of units of power that is reserved in combination with the device instance 440 help identify an allocable amount (e.g., allocable units 460). A power management scheme supports generating and reserving power based on corresponding electrical components and electricity processing techniques associated with the power device. The power manager 410 and the remote power interface 430 can communicate corresponding notifications to actuate one or more physical devices within the datacenter and a power grid, respectively. The notifications initiate operation processes for the processing the amount of allocated units of power.
Turning now to
At block 530, a request having requested units of power is received. The request can be received from a remote power interface. Upon receiving the request, the power manager can communicate operation instructions to the power device for an amount of units of power to be generated and an amount of units of power to be reserved, based at least in part on the device instance and the minimum threshold reserve of the power associated with the datacenter. The power being generated is generated based on the power device throttling usage of the power such that an unused portion of the power is made available for consumption via the datacenter. Reserving the power is based on an internal component or an external component associated with the byproduct device.
At block 540, an amount of allocated units for the request based on an amount of units of the power that is being generated and an amount of units of the power that is reserved is communicated. The amount of units of the power being generated and the amount of units of the power being reserved is based at least in part on the device instance and a minimum threshold reserve of the power associated with operating the datacenter. The power generated is based on an Application Programming Interface that supports a protocol for processing powers based on the power device in relation to power and a reserve of power.
Turning now to
At block 630, an amount of allocated units of the power for the request based on an amount of units of the power that is being generated and an amount of units of the power that is reserved is received. The amount of units of the power being generated and the amount of units of the power being reserved is based at least in part on the device instance and the minimum threshold reserve of the power associated with the datacenter. A notification to actuate one or more physical devices within an operating infrastructure that is the operating environment of the remote device is communicated to initiate operation processes for the processing the amount of allocated units of the power. The power grid is a smart grid that operates based on a grid connect system to opportunistically balance the power grid based on the datacenter power usage.
Datacenter Operations Optimization System
With reference to
In operation, the byproduct resource manager 710 receives byproduct device data (e.g., byproduct device data 722) for a plurality of byproduct devices (e.g., plurality of byproduct devices 720) associated with a datacenter 720A associated with a co-located operating infrastructure (e.g., 720B) that is an operating environment that shares physical resources with the datacenter. The datacenter 720A is shown with greenhouse 720 as an exemplary co-located operating infrastructure. The byproduct resource manager is responsible for generating, based on the byproduct device data, a plurality of device instances (e.g., device instance 740) having a plurality of attributes associated with the byproduct devices. The plurality of device instances includes a generation capacity of units of one or more physical resources and reserve units of the one or more physical resources. A device instance is a computed representation of the byproduct devices in relation to the one or more physical resources and a reserve of the one or more physical resources, the device instance supports determining allocable units of the one or more physical resources.
The byproduct resource manager communicates operation instructions (e.g., operation instructions 724) to the byproduct devices for an amount of units of the one or more physical resources to be generated and an amount of units of the one or more physical resources to be reserved, based on the plurality of device instances and a minimum threshold reserve of the one or more physical resources associated with operating the datacenter with the co-located operating infrastructure. The minimum threshold reserve can be retrieved from the byproduct threshold manager 716.
The byproduct resource manager 110 automatically updates an amount of allocable units of the one or more physical resources based on an amount of the one or more physical resources that are being generated and an amount of units of the one or more physical resources that are reserved. The one or more physical resources to be generated are generated based on the byproduct device producing the one or more physical resources. A physical resource can be produced by transmitting the physical resource from a generation source to be made available for consumption via the datacenter or the byproduct device throttling usage of the one or more physical resources such that an unused portion of the one or more physical resource is made available for consumption via the datacenter. Reserving the one or more physical resources is based on an internal component or an external component associated with the byproduct devices in the datacenter and co-located operating infrastructure.
The remote resource interface is responsible for generating a request having requested units of the one or more physical resources. The byproduct resource manager can receive the request and generate an operation instance (e.g., operation instance 750) for processing the request. The remote resource interface 730 can communicate the requested units 732 to receive allocated units (e.g., allocated units 734) of the one or more physical resources from the byproduct resource manager 710. The remote resource interface 730 receives an amount of allocated units for the request based on an amount of units of power that is being generated and an amount of units of power that is reserved. In particular, the amount of units of power that is being generated and an amount of units of the one or more physical resources that is reserved in combination with the device instance 740 help identify an allocable amount (e.g., allocable units 760). The byproduct resource manager 710 and the remote resource interface 730 can communicate corresponding notifications to actuate one or more physical devices within the datacenter and an operating environment for the remote resource interface, respectively. The notifications initiate operation processes for the processing the amount of allocated units of the one or more physical resources.
The remote datacenter manager of remote datacenter 730C is responsible for generating a request having requested units for compute resources. The remote datacenter manager can communicate the request to the byproduct resource manager 710 (e.g., compute resource manager—not shown) in order to receive an amount of allocated units of compute resources based on an amount of units of the one or more physical resources that are being generated and an amount of units of the one or more physical resources that are reserved. The remote datacenter manager can transmit a compute workload to the datacenter to utilize the amount of allocated units of compute resources that are based on the availability of physical resources at the datacenter.
Turning now to
At block 830, a request having requested units of the physical resource is received. The request can be received from a remote resource interface. The request can be received based on APIs that are supported at the remote resource interface such that the request is provided in based on protocols of the API. Upon receiving the request, the byproduct resource manager can communicate operation instructions to the byproduct device for an amount of units of the one or more physical resources to be generated and an amount of units of the physical resource to be reserved, based at least in part on the device instance and the minimum threshold reserve of the one or more physical resources associated with the datacenter. The one or more physical resource being generated is generated based on the byproduct device throttling usage of the physical resource such that an unused portion of the one or more physical resource is made available for consumption via the datacenter. Reserving the one or more physical resource is based on an internal component or an external component associated with the byproduct device.
At block 840, an amount of allocated units for the request based on an amount of units of the one or more physical resource that is being generated and an amount of units of the physical resource that are reserved, is communicated. The amount of units of the one or more physical resources being generated and the amount of units of the physical resource being reserved is based at least in part on the device instance and a minimum threshold reserve of the one or more physical resource associated with operating the datacenter. The byproduct resource manager automatically updates an amount of allocable units of the one or more physical resources based on an amount of the one or more physical resources that are being generated and an amount of units of the one or more physical resources that are reserved.
Turning now to
At block 930, an amount of allocated units of the physical resource for the request based on an amount of units of the one or more physical resources that is being generated and an amount of units of the physical resource that is reserved is received. The amount of units of the physical resources being generated and the amount of units of the one or more physical resource being reserved is based at least in part on the device instance and the minimum threshold reserve of the one or more physical resources associated with the datacenter. A notification to actuate one or more physical devices within an operating infrastructure that is the operating environment of the remote device is communicated to initiate operation processes for the processing the amount of allocated units of the physical resource. The operating infrastructure is selected from one of the following: a power grid, a co-located agricultural facility, a manufacturing facility, a building, and a remote datacenter. The byproduct resource manager upon receiving the request communicates operation instructions to the byproduct device for an amount of units of the physical resource to be generated and the amount of units of the physical resource to be reserved, based on the device instance and the minimum threshold reserve of the physical resource associated with operating the datacenter.
Infrastructure Management System
With reference to
In operation, the byproduct resource manager 1010 (or infrastructure resource manager) receives infrastructure device data (e.g., infrastructure device data 1022) for an infrastructure device (e.g., infrastructure device 1030) associated with a physical resource of an infrastructure. The infrastructure (e.g., infrastructure 1020A) can be selected form one of the following: a power grid, a co-located agricultural facility, a manufacturing facility, a building, and a remote datacenter. An operation instance (e.g., operation instance 1050) is generated based on the infrastructure device data. The operation instance includes a plurality of attributes associated with the infrastructure device. The operation instance also includes requested units that indicate an amount of a physical resource is needed to perform an infrastructure device operation. The byproduct resource manager 1010 communicates operation instructions (e.g., operation instructions 1024) to one or more additional infrastructure devices (e.g., additional infrastructure devices 1030B) for an amount of units of the physical resource to be generated and an amount of units of the physical resource to be reserved. The operating instructions are based on the operation instance and a minimum threshold reserve of the physical resource associated with operating the infrastructure. The byproduct resource manager 1010 determines an amount of allocable units (e.g., allocable units 1060) and an amount of allocated units of the physical resource based in part on the operation instance and an amount of units of the physical resource that is being generated and the amount of units of the physical resource that is reserved and then communicates operation instructions (e.g., operation instructions 1032) to the infrastructure device. The operation instructions include the amount of allocated units of the physical resource. The byproduct resource manager may be responsible to generate, based on the infrastructure device data, a device instance (e.g., device instance 1040) having a plurality of attributes associated with the infrastructure device, the device instance comprising a generation capacity of units of a physical resource and reserve units of the physical resource, wherein the device instance is a computed representation of the byproduct device in relation to the physical resource and a reserve of the physical resource, the device instance supports determining allocable units of the physical resource.
The infrastructure resource manager automatically updates an amount of allocable units of the physical resource based on the amount of units of the physical resource that is being generated and an amount of units of the physical resource that is reserved. The physical resource to be generated is generated based on the one or more additional infrastructure devices producing the physical resource in part by transmitting the physical resource from a generation source to be made available for consumption via the datacenter or the one or more additional infrastructure devices throttling usage of the physical resource such that an unused portion of the physical resource is made available for consumption via the datacenter.
An infrastructure device is configured to communicate the infrastructure device data that is used in determining the amount of the requested physical resource units need for the anticipated infrastructure device operation, receive the amount of allocated units for the infrastructure device operation; and initiate processing of the amount of allocated units for the performing the infrastructure device operation. By way of example, the infrastructure device is an electrical appliance that is associated with the infrastructure operating as a building facility and the one or more additional infrastructures device are computing devices within the building. In another example, the infrastructure device is an electrical grid device that is associated with the infrastructure operating based on an electric grid, a grid connector, and a datacenter, and the one or more additional infrastructure devices are power devices in the datacenter.
As discussed with reference to datacenter optimization operations system, the infrastructure management system can further include support a remote datacenter manager that is responsible for generating a request having requested units for compute resources. The remote datacenter manager can communicate the request to the byproduct resource manager 1010 (e.g., compute resource manager—not shown) in order to receive an amount of allocated units of compute resources based on an amount of units of the one or more physical resources that are being generated and an amount of units of the one or more physical resources that are reserved. The remote datacenter manager can transmit a compute workload to the datacenter to utilize the amount of allocated units of compute resources that are based on the availability of physical resources at the datacenter.
Turning now to
At block 1140, an amount of allocated units of the physical resources is determined based in part on the instance and an amount of units of the physical resource that is being generated and the amount of units of the physical resource that is reserved. The physical resource being generated is generated based on the byproduct device producing the physical resource in part by transmitting the physical resource from a generation source to be made available for consumption via the datacenter. In the alternative, the physical resource being generated is generated based on the byproduct device throttling usage of the physical resource such that an unused portion of the physical resource is made available for consumption via the datacenter. At block 1150, operation instructions to the infrastructure device are communicated. The operation instructions include the amount of allocated units of the physical resource. A notification to actuate one or more physical devices within the infrastructure is communicated to initiate operation processes for processing the amount of allocated units of the physical resource.
Turning now to
Computing Environments and Context
Having identified various components of the distributed computing environments in
Other arrangements and elements (e.g., machines, interfaces, functions, orders, and groupings of functions, etc.) can be used in addition to or instead of those shown, and some elements may be omitted altogether. Further, many of the elements described herein are functional entities that may be implemented as discrete or distributed components or in conjunction with other components, and in any suitable combination and location. Various functions described herein as being performed by one or more entities may be carried out by hardware, firmware, and/or software. For instance, various functions may be carried out by a processor executing instructions stored in memory.
With further reference to the resource management system, embodiments described herein can improve datacenter operations based on a resource management framework. The integrated components of the resource management framework refer to the hardware architecture and software framework that support functionality using the fiber optic cable network testing. The hardware architecture refers to physical components and interrelationships thereof and the software framework refers to software providing functionality that can be implemented with hardware operated on a device. The end-to-end software-based resource management framework can operate within the resource management system components to operate computer hardware to provide functionality described herein. As such, the components can manage resources and provide services for the resource management functionality. Any other variations and combinations thereof are contemplated with embodiments of the present invention.
By way of example, the resource management framework can include an API library that includes specifications for routines, data structures, object classes, and variables may support the interaction the hardware architecture of the device and the software framework of the testing platform. These APIs include configuration specifications for the resource management system such that the components therein can communicate with each other, as described herein.
Having briefly described an overview of embodiments of the present invention, an exemplary operating environment in which embodiments of the present invention may be implemented is described below in order to provide a general context for various aspects of the present invention. Referring initially to
The invention may be described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program modules, being executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program modules including routines, programs, objects, components, data structures, etc. refer to code that perform particular tasks or implement particular abstract data types. The invention may be practiced in a variety of system configurations, including hand-held devices, consumer electronics, general-purpose computers, more specialty computing devices, etc. The invention may also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network.
With reference to
Computing device 1300 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computing device 1300 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media may comprise computer storage media and communication media.
Computer storage media include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device 100. Computer storage media excludes signals per se.
Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.
Memory 1312 includes computer storage media in the form of volatile and/or nonvolatile memory. The memory may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical-disc drives, etc. Computing device 1300 includes one or more processors that read data from various entities such as memory 1312 or I/O components 1320. Presentation component(s) 1316 present data indications to a user or other device. Exemplary presentation components include a display device, speaker, printing component, vibrating component, etc.
I/O ports 1318 allow computing device 1300 to be logically coupled to other devices including I/O components 1320, some of which may be built in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, etc.
Referring now to
Data centers can support the distributed computing environment 1400 that includes the cloud computing platform 1410, rack 1420, and node 1430 (e.g., computing devices, processing units, or blades) in rack 1420. The system can be implemented with a cloud computing platform 1410 that runs cloud services across different data centers and geographic regions. The cloud computing platform 1410 can implement a fabric controller 1440 component for provisioning and managing resource allocation, deployment, upgrade, and management of cloud services. Typically, the cloud computing platform 1410 acts to store data or run service applications in a distributed manner. The cloud computing infrastructure 1410 in a data center can be configured to host and support operation of endpoints of a particular service application. The cloud computing infrastructure 1410 may be a public cloud, a private cloud, or a dedicated cloud.
The node 1430 can be provisioned with a host 1450 (e.g., operating system or runtime environment) running a defined software stack on the node 130. Node 1430 can also be configured to perform specialized functionality (e.g., compute nodes or storage nodes) within the cloud computing platform 1410. The node 1430 is allocated to run one or more portions of a service application of a tenant. A tenant can refer to a customer utilizing resources of the cloud computing platform 1410. Service application components of the cloud computing platform 1410 that support a particular tenant can be referred to as a tenant infrastructure or tenancy. The terms service application, application, or service are used interchangeably herein and broadly refer to any software, or portions of software, that run on top of, or access storage and compute device locations within, a datacenter.
When more than one separate service application is being supported by the nodes 1430, the nodes may be partitioned into virtual machines (e.g., virtual machine 1452 and virtual machine 1454). Physical machines can also concurrently run separate service applications. The virtual machines or physical machines can be configured as individualized computing environments that are supported by resources 1460 (e.g., hardware resources and software resources) in the cloud computing platform 1410. It is contemplated that resources can be configured for specific service applications. Further, each service application may be divided into functional portions such that each functional portion is able to run on a separate virtual machine. In the cloud computing platform 1410, multiple servers may be used to run service applications and perform data storage operations in a cluster. In particular, the servers may perform data operations independently but exposed as a single device referred to as a cluster. Each server in the cluster can be implemented as a node.
Client device 1480 may be linked to a service application in the cloud computing platform 1410. The client device 1480 may be any type of computing device, which may correspond to computing device 1000 described with reference to
Having described various aspects of the distributed computing environment 1400 and cloud computing platform 1410, it is noted that any number of components may be employed to achieve the desired functionality within the scope of the present disclosure. Although the various components of
The subject matter of embodiments of the disclosure is described with specificity herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the terms “step” and/or “block” may be used herein to connote different elements of methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described.
Embodiments described herein may be combined with one or more of the specifically described alternatives. In particular, an embodiment that is claimed may contain a reference, in the alternative, to more than one other embodiment. The embodiment that is claimed may specify a further limitation of the subject matter claimed.
For detailed discussion purposes, the resource management system 100 is described as a mechanism associated with an infrastructure (e.g., datacenter 120A running the plurality of devices 120). A mechanism as used herein refers to any device, process, or service or combination thereof. A mechanism may be implemented using components as hardware, software, firmware, a special-purpose device, or any combination thereof. A mechanism may be integrated into a single device or it may be distributed over multiple devices. The various components of a mechanism may be co-located or distributed. The mechanism may be formed from other mechanisms and components thereof. The components of the resource management system 100 facilitate providing functionality described herein.
For purposes of a detailed discussion above, embodiments are described with reference to distributed computing devices, components, and a resource management system components. Components can be configured to perform novel aspects of embodiments described herein, where “configured to” includes components that are programmed to perform particular tasks or implement particular abstract data types using computer code. It is contemplated that the methods described herein can be performed in different types of operating environments having alternate configurations of the functional components described herein. As such, the embodiments described herein are merely exemplary, and it is contemplated that the techniques may be extended to other implementation contexts.
For purposes of this disclosure, the word “including” has the same broad meaning as the word “comprising,” and the word “accessing” involves “receiving,” “referencing,” or “retrieving.” In addition, words such as “a” and “an,” unless otherwise indicated to the contrary, include the plural as well as the singular. Further the word “communicating” has the same broad meaning as the word “receiving,” or “transmitting” facilitated by software or hardware-based buses, receivers, or transmitters” using communication media described herein. Thus, for example, the constraint of “a feature” is satisfied where one or more features are present. Also, the term “or” includes the conjunctive, the disjunctive, and both (a or b thus includes either a or b, as well as a and b).
Embodiments presented herein have been described in relation to particular embodiments which are intended in all respects to be illustrative rather than restrictive. Alternative embodiments will become apparent to those of ordinary skill in the art to which the present invention pertains without departing from its scope.
From the foregoing, it will be seen that this invention is one well adapted to attain all the ends and objects hereinabove set forth together with other advantages which are obvious and which are inherent to the structure.
It will be understood that certain features and sub-combinations are of utility and may be employed without reference to other features or sub-combinations. This is contemplated by and is within the scope of the claims.
This application is a continuation of U.S. application Ser. No. 15/199,961, filed Jun. 30, 2016, which is related by subject matter to the subject matter disclosed in the following commonly assigned applications: U.S. application Ser. No. 15/199,959, entitled “DATACENTER BYPRODUCT MANAGEMENT INTERFACE SYSTEM” U.S. application Ser. No. 15/199,963, entitled “DATACENTER OPERATIONS OPTIMIZATION SYSTEM” and U.S. application Ser. No. 15/199,964, entitled “INFRASTRUCTURE RESOURCE MANAGEMENT SYSTEM.” Each of the aforementioned applications is herein incorporated by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 8429630 | Nickolov et al. | Apr 2013 | B2 |
| 8904209 | Davis et al. | Dec 2014 | B2 |
| 8983669 | Forbes, Jr. | Mar 2015 | B2 |
| 9063525 | Sanders et al. | Jun 2015 | B2 |
| 9207698 | Forbes, Jr. | Dec 2015 | B2 |
| 9515491 | Bhageria et al. | Dec 2016 | B2 |
| 9519878 | Sankar | Dec 2016 | B2 |
| 9568910 | Drees et al. | Feb 2017 | B2 |
| 9740227 | Forbes, Jr. | Aug 2017 | B2 |
| 9754330 | Kitagishi | Sep 2017 | B2 |
| 9960637 | Sanders et al. | May 2018 | B2 |
| 9985848 | Ward, Jr. | May 2018 | B1 |
| 10057185 | Burgin et al. | Aug 2018 | B1 |
| 20080184230 | Leech et al. | Jul 2008 | A1 |
| 20100218108 | Crabtree et al. | Aug 2010 | A1 |
| 20120054125 | Clifton et al. | Mar 2012 | A1 |
| 20150134135 | Wong et al. | May 2015 | A1 |
| 20160370843 | Gatson et al. | Dec 2016 | A1 |
| 20180004265 | James et al. | Jan 2018 | A1 |
| 20180006964 | Joppa et al. | Jan 2018 | A1 |
| 20180006965 | Rawlings et al. | Jan 2018 | A1 |
| 20180006966 | James et al. | Jan 2018 | A1 |
| 20180188790 | Griffith et al. | Jul 2018 | A1 |
| Entry |
|---|
| “Final Office Action Issued in U.S. Appl. No. 15/199,964”, dated Jan. 23, 2019, 11 Pages. |
| “Non Final Office Action Issued in U.S. Appl. No. 15/199,964”, dated Jul. 6, 2018, 19 Pages. |
| Wierman, et al., “Opportunities and Challenges for Data Center Demand Response”, In Proceedings of International Green Computing Conference, Nov. 3, 2014, 10 Pages. |
| “Non Final Office Action Issued in U.S. Appl. No. 15/199,959”, dated Feb. 2, 2018, 14 Pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20180321726 A1 | Nov 2018 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15199961 | Jun 2016 | US |
| Child | 16038449 | US |
