The present disclosure relates to the field of computing technologies, and particularly relates to the field of cloud resources.
In the applications of virtual machines or virtual machine clusters and so on, the main functions of resource management and scheduling include establishing and maintaining in real time running node resources of the system, heterogeneous features of resource objects and resource state management, providing state and scheduling management of interactive resources and resource objects, and processing scheduling request services of running node resources and object resources.
The cloud resource technology can implement virtualization and desktop migration of physical resources, realizing virtualization and basic resource management, allocation and use. In the existing art, general resource management and scheduling merely involve query and calculation, as well as allocation and use, of available resources, or rescheduling and migration for resource users exceeding a node load, aiming merely at maintaining resource availability and preventing the node from operating in an overloading state.
However, in the actual application process, resources for practical productions and applications are typically deployed in a multi-node or cluster manner, and the use of resource is dynamically changed, which can easily cause decentralized use of the node resources that leads to decentralized and irregular occupation and release of the node resources, resulting in a low resource utilization rate, waste of the node resources, and high use cost.
The present disclosure provides a resource scheduling method and apparatus, a computer device, and a readable medium.
In a first aspect, an embodiment of the present disclosure provides a resource scheduling method, including: calculating first resource parameters of respective nodes and second resource parameters of respective resource users at the respective nodes; calculating resource utilization rates of the respective nodes according to the first resource parameters of the respective nodes; calculating a distribution parameter of the respective resource users according to the first resource parameters of the respective nodes and the second resource parameters of the respective resource users; and in response to at least the resource utilization rates of the respective nodes or the distribution parameter of the respective resource users satisfying a resource aggregation condition, determining a resource user to be migrated and a target node, and changing a node, to which the resource user to be migrated belongs, to the target node; wherein the resource user to be migrated is one or more of the respective resource users, and the target node is one or more of the respective nodes.
In another aspect, an embodiment of the present disclosure further provides a resource scheduling apparatus, including: a calculation module and a resource scheduling module, wherein the calculation module is configured to calculate first resource parameters of respective nodes and second resource parameters of respective resource users at the respective nodes; calculate resource utilization rates of the respective nodes according to the first resource parameters of the respective nodes; and calculate a distribution parameter of the respective resource users according to the first resource parameters of the respective nodes and the second resource parameters of the respective resource users; wherein the resource scheduling module is configured to, in response to at least the resource utilization rates of the respective nodes or the distribution parameter of the respective resource users satisfying a resource aggregation condition, determine a resource user to be migrated and a target node, and change a node, to which the resource user to be migrated belongs, to the target node; wherein the resource user to be migrated is one or more of the respective resource users, and the target node is one or more of the respective nodes.
In another aspect, an embodiment of the present disclosure further provides a computer device, including: one or more processors; and a storage means having one or more programs stored thereon which, when executed by the one or more processors, cause the one or more processors to perform the resource scheduling method as described above.
In another aspect, an embodiment of the present disclosure further provides a computer-readable medium having a computer program stored thereon which, when executed, causes the resource scheduling method as described above to be implemented.
Example embodiments will be described more sufficiently below with reference to the accompanying drawings, which, however, may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.
As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.
Terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the present disclosure. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that as used herein, the terms “comprise” and/or “consist of . . . ” specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Embodiments described herein may be described with reference to plan and/or sectional views in idealized representations of the present disclosure. Accordingly, the example illustrations may be modified in accordance with manufacturing techniques and/or tolerances. Therefore, the embodiments are not limited to the embodiments shown in the drawings, but further include modifications of configurations formed based on a manufacturing process. Therefore, the regions illustrated in the figures have schematic properties, and the shapes of the regions shown in the figures illustrate specific shapes of regions of elements, but are not intended to be limitative.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the existing art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
An embodiment of the present disclosure provides a resource scheduling method applied in a system shown in
With the resource management and scheduling method provided in the embodiments of the present disclosure, resources can be dynamically scheduled in a cloud resource environment, thereby improving the resource utilization rate. The embodiments of the present disclosure may apply the following policies:
Current resource management policies in the existing art involve allocating and using resources under the condition that the resource utilization rate and the demand are satisfied, and releasing resources that are not used, which follows availability and extensive management in the aspect of resource management. By contrast, in the embodiments of the present disclosure, resource domains are classified and divided according to needs, resources of multiple nodes in the resource domains are calculated and planned as a whole, virtual resources obtained by virtualizing the physical resources are scheduled uniformly, and resource management and calculation are performed from multiple dimensions including the node load, resource bearing capacity, network bandwidth, time delay and so on. Each resource user performs resource management and calculation from multiple dimensions including the required resources, resource use conditions and so on, and uses the monitored scheduling period, or operations of the resource user such as logging in, logging out, closing a desktop, adding or deleting resources or the like, as a condition for triggering resource scheduling.
The resources are divided into two categories, i.e., computing resources and storage resources. The computing resources can be dynamically adjusted and changed according to the needs of the resource user, and are independent of specific objects; and the storage resources belong to fixed resources, and are used for storing data information. The resource scheduler monitors, manages, allocates, and operates related information of the physical device resources and the virtualized resources. The configuration management layer makes a response to, according to a configuration policy, resource related requests or operations of a resource user authenticated by the user center. The policy center may set monitoring, detection and thresholds differently according to the requirements of the resource user, and perform real-time multi-dimensional detection, analysis and overall calculation when the resource user are using the resources, to implement real-time dynamic resource management and scheduling.
As shown in
At operation S21, calculating first resource parameters of respective nodes and second resource parameters of respective resource users at the respective nodes.
A cluster or resource domain includes a plurality of nodes, each of which is deployed with various resources for use by one or more resource users. In this operation, a cluster or resource domain as a unit, a first resource parameter of each node in the cluster or resource domain is calculated, and a second resource parameter is calculated for each resource user at each node.
At operation S22, calculating resource utilization rates of the respective nodes according to the first resource parameters of the respective nodes.
The first resource parameter may include used resources and total available resources. The used resources of the first resource parameter are represented by a feature vector Ci, and the total available resources of the first resource parameter are represented by a feature vector CSi, where i represents a node ID.
In this operation, a resource utilization rate of the first resource parameter for a node may be calculated from the used resources of the first resource parameter and the total available resources of the first resource parameter of the node. The node i has a resource utilization rate of γi, and
At operation S23, calculating a distribution parameter of the respective resource users according to the first resource parameters of the respective nodes and the second resource parameters of the respective resource users.
The second resource parameter of the resource user may be represented by a feature vector Uj, where j represents a resource user ID. In this operation, the distribution parameter of the respective resource users may be calculated according to the used resources of the first resource parameters of the respective nodes and the second resource parameters of the respective resource users, where the distribution parameter reflects a distribution condition of the resource users.
At operation S24, in response to at least the resource utilization rates of the respective nodes or the distribution parameter of the respective resource users satisfying a resource aggregation condition, determining a resource user to be migrated and a target node, and changing a node, to which the resource user to be migrated belongs, to the target node.
The resource user to be migrated is one or more of the respective resource users, and the target node is one or more of the respective nodes. In this operation, it is judged whether the resource utilization rates of the respective nodes and/or the distribution parameter of the respective resource users satisfy the resource aggregation condition, and if the resource aggregation condition is satisfied, a resource user to be migrated from the current node, and a target node used to receive the resource user to be migrated (i.e., a node to which the resource user to be migrated is migrated) are determined, and then, the node to which the resource user to be migrated belongs is changed to the target node, thereby implementing resource scheduling and aggregation among the respective nodes. The resource user to be migrated and the target node may be represented by a set of resource users to be migrated which are migrated to target nodes.
In some embodiments, the distribution parameter of the resource user may be a distribution dispersion rate (or distribution dispersion degree) of the resource users. In judgment of the resource aggregation condition, the judgment may be made solely according to the resource utilization rate of each node and a preset resource utilization rate threshold, or made solely according to the distribution dispersion rate of the resource users and a preset resource user distribution dispersion rate threshold, or made according to a combination of the resource utilization rate of each node, a preset resource utilization rate threshold, the distribution dispersion rate of the resource users, and a preset resource user distribution dispersion rate threshold. It should be noted that a higher resource utilization rate of the node leads to a lower distribution dispersion rate of the resource users, and vice versa.
The embodiment of the present disclosure provides a resource scheduling method, including: calculating first resource parameters of respective nodes and second resource parameters of respective resource users at the respective nodes; calculating resource utilization rates of the respective nodes according to the first resource parameters of the respective nodes; calculating a distribution parameter of the respective resource users according to the first resource parameters of the respective nodes and the second resource parameters of the respective resource users; and in response to at least the resource utilization rates of the respective nodes or the distribution parameter of the respective resource users satisfying a resource aggregation condition, determining a resource user to be migrated and a target node, and changing a node, to which the resource user to be migrated belongs, to the target node. In the embodiment of the present disclosure, resource deployment at the nodes, and demands and use of the resources by the resource users are calculated and analyzed to migrate the resource users among the nodes and thus implement resource aggregation, and resources are scheduled as a whole from the whole cluster perspective at a global level, thereby improving the resource utilization rate of each node, avoiding resource waste at the nodes, and reducing the resource use cost.
In some embodiments, determining the resource user to be migrated and the target node (i.e., operation S24) includes: determining the resource user to be migrated and the target node according to the resource utilization rates of the respective nodes, the distribution parameter of the respective resource users, a preset resource utilization rate threshold and a preset resource user distribution parameter threshold, the first resource parameters of the respective nodes, and the second resource parameters of the respective resource users.
To obtain more precise resource use conditions and resource demands, the first resource parameters of the nodes and/or the second resource parameters of the resource users may be divided into a plurality of types. Therefore, in some embodiments, in the case where the first resource parameters comprise least two types, calculating the first resource parameters of the respective nodes (i.e., operation S21) includes: calculating the first resource parameters of the respective nodes according to the first resource parameters of each type and a corresponding first factor. The first factor represents an importance degree of each type of the first resource parameters. In other words, the first resource parameters of the nodes should be calculated with reference to all types of first resource parameters.
In some embodiments, in the case where the second resource parameters comprise least two types, calculating the second resource parameters of the respective resource users at the respective nodes (i.e., operation S21) includes: calculating the second resource parameters of the respective resource users at the respective nodes according to the second resource parameters of each type and a corresponding second factor. The second factor represents an importance degree of each type of the second resource parameters. In other words, the second resource parameters of the resource users should be calculated with reference to all types of second resource parameters.
In some embodiments, the first factor includes at least one of: a first weight or a first operational relationship.
In some embodiments, the second factor includes at least one of: a second weight or a second operational relationship.
The first and/or second operational relationship may include an algebraic relationship (such as power, radication, logarithm, or the like), or may include an arithmetic relationship such as derivative operation, integral operation, convolution operation, rounding operation, or the like. Taking calculation of the first resource parameter of a node as an example, if the first resource parameter includes different types of first resource parameters, such as a first resource parameter 1 and a first resource parameter 2, and the first factor of the first resource parameter 1 and the first factor of the first resource parameter 2 are both an operational relationship of power, then the first resource parameter of the node can be calculated from a power of the first resource parameter 1 and a power of the first resource parameter 2. Exemplarily, first resource parameter of the node=(first resource parameter 1)n+(first resource parameter 2)n. If the first factor of the first resource parameter 1 and the first factor of the first resource parameter 2 are both a first weight, the first resource parameter 1 corresponds to a first weight W1, and the first resource parameter 2 corresponds to a first weight W2, then exemplarily, first resource parameter of the node=(first resource parameter 1)*W1+(first resource parameter 2)*W2.
In some embodiments, the types of the first resource parameters may include: load, resource bearing capacity, network bandwidth, and time delay. In calculation of the first resource parameters of the nodes, at least two of the above types of first resource parameters are used.
In the embodiments of the present disclosure, the load generally refers to a resource use condition; the resource bearing capacity generally refers to total available resources, and may include, for example: a disk space size for measuring a bearing capacity of a disk, an I/O read-write rate, and a CPU clock speed and the number of cores for measuring a computing capacity of the CPU. It should be noted that the types of the first resource parameters are not limited to those listed above, and may be selected and determined according to the actual service scenario. For example, for video on-demand cloud resources, only the disk size, the I/O read-write rate, and the network bandwidth may be of interest, and therefore, two types of first resource parameters, i.e., the disk bearing capacity and the network bandwidth, may be selected.
In some embodiments, the types of the second resource parameters of the respective resource users may include: resource usage duration, resource usage frequency, bandwidth requirements of a data transmission network, resource requirement type, and resource utilization rate. In calculation of the second resource parameters of the resource users, at least two of the above types of second resource parameters are used.
In some embodiments, the respective nodes belong to a same and single resource domain, and calculating the first resource parameters of the respective nodes and the second resource parameters of the respective resource users at the respective nodes (i.e., operation S21) includes: calculating first resource parameters of respective nodes and second resource parameters of respective resource users at the respective nodes in the resource domain. In other words, in the case that there is only one resource domain, all nodes belong to the same resource domain, and the first resource parameters of all nodes and the second resource parameters of the resource users thereof in the whole resource domain are calculated. Accordingly, the calculated distribution parameter of the respective resource users also reflects a distribution condition of the resource users at respective nodes in the resource domain, and the resource user to be migrated is migrated among the nodes in the resource domain, to implement resource scheduling within the resource domain.
To further improve the resource utilization rate and the convenience in resource use, nodes providing the same type of resources may be divided into the same resource domain. It should be noted that, in the embodiments of the present disclosure, the types of resources may include, but are not limited to, computing resources, graphics resources, and so on.
Accordingly, in some embodiments, the respective nodes each belong to one or more of a plurality of resource domains divided according to types of resources, respective nodes in the same resource domain are deployed with the same type of resources, and calculating the first resource parameters of the respective nodes and the second resource parameters of the respective resource users at the respective nodes (i.e., operation S21) includes: calculating first resource parameters of respective nodes and second resource parameters of respective resource users at the respective nodes in the same resource domain.
In other words, in the case of a plurality of resource domains, different nodes may belong to different resource domains, or different types of resources may be deployed in the same node, so that the same node may belong to different resource domains. In either case, the first resource parameters of the nodes and the second resource parameters of the resource users are calculated in units of resource domains. Accordingly, the calculated distribution parameter of resource users also reflects a distribution condition of the resource users at the respective nodes in the same resource domain, and the resource user to be migrated is migrated among the nodes in the same resource domain, to implement resource scheduling within the same resource domain.
In the case of multiple types of first resource parameters and multiple types of second resource parameters, the used resources of the first resource parameter Ci of the node may be represented as:
The total available resources CSi of the first resource parameter of the node may be expressed as:
The second resource parameter Uj of the resource user may be expressed as:
The resource utilization rate γd of a certain resource domain d may be calculated by:
In the resource domain d, the distribution dispersion rate of the resource users may be expressed as yd, and exemplarily, yd may be calculated by:
For a certain resource domain d, the set of resource users to be migrated which are migrated to target nodes may represented by ad, and exemplarily, ad may be calculated by:
It should be noted that in the resource domain d, the distribution dispersion rate of the resource users yd and the set ad of resource users to be migrated which are migrated to target nodes include, but are not limited to, the above equations, and any solution capable of determining the distribution discrete rate, the resource users to be migrated and the target node is within the scope of the present disclosure.
In the embodiments of the present disclosure, a triggering condition for calculating the first resource parameters of the respective nodes and the second resource parameters of the respective resource users at the respective nodes may include: periodically triggering (i.e., reaching a preset trigger time), or event triggering (i.e., occurrence of a trigger event).
Accordingly, in some embodiments, calculating the first resource parameters of the respective nodes and the second resource parameters of the respective resource users at the respective nodes (i.e., operation S21) includes:
In a scenario of a cloud video resource on demand, a video on demand cloud resource server receives event-triggered resource scheduling from a third-party gateway or other devices, and correspondingly, if no user requests a certain video content within a preset time (e.g., 24 continuous hours), the first resource parameters of respective nodes and the second resource parameters of respective resource users at the respective nodes are calculated to trigger video resource scheduling. It should be noted that the above trigger event is only exemplary, and may be defined according to the actual service scenario.
With the resource scheduling method provided in the embodiments of the present disclosure, cloud resources can be uniformly managed and planned, resource applications of the resource users can be reasonably allocated and scheduled, and unified management of resource release can be implemented. Dynamic management of the resources is performed according to a set policy, and dynamic management, scheduling and allocation of the resources are implemented through real-time automatic monitoring, calculation, migration, aggregation, resource recovery and release, thereby achieving maximum resource utilization.
By classifying the resources, demands of the resource users are divided into one or more resource domains through policy setting. Then, the resource domain is taken as a unit management and scheduling body for multi-dimensional calculation on the resources of the nodes and the resource users in the resource domain, and unified data analysis, processing and overall planning are performed within the resource domain, to determine a target node to receive aggregation migration and resource users to be aggregated and migrated, thereby completing the aggregation and migration of resources, and improving the utilization rate of the node resources which are otherwise used at the nodes and by the resource users discretely and irregularly through migration and aggregation.
Based on the same technical concept, an embodiment of the present disclosure further provides a resource scheduling apparatus which, as shown in
The resource scheduling module 102 is configured to, in response to at least the resource utilization rates of the respective nodes or the distribution parameter of the respective resource users satisfying a resource aggregation condition, determine a resource user to be migrated and a target node, and change a node, to which the resource user to be migrated belongs, to the target node. The resource user to be migrated is one or more of the respective resource users, and the target node is one or more of the respective nodes.
In some embodiments, the resource scheduling module 102 is configured to determine the resource user to be migrated and the target node according to the resource utilization rates of the respective nodes, the distribution parameter of the respective resource users, a preset resource utilization rate threshold and a preset resource user distribution parameter threshold, the first resource parameters of the respective nodes, and the second resource parameters of the respective resource users.
In some embodiments, the first resource parameters comprise least two types, and the calculation module 101 is configured to calculate first resource parameters of respective nodes according to the first resource parameters of each type and a corresponding first factor that represents an importance degree of each type of the first resource parameters; and/or
In some embodiments, the first factor includes at least one of: a first weight or a first operational relationship; and/or
In some embodiments, the types of the first resource parameters include: load, resource bearing capacity, network bandwidth, and time delay; and/or the types of the second resource parameters of the respective resource users include: resource usage duration, resource usage frequency, bandwidth requirements of a data transmission network, resource requirement type, and resource utilization rate.
In some embodiments, the respective nodes belong to a same and single resource domain, and the calculation module 101 is configured to calculate first resource parameters of respective nodes and second resource parameters of respective resource users at the respective nodes in the resource domain.
In some embodiments, the respective nodes each belong to one or more of a plurality of resource domains divided according to types of resources, respective nodes in the same resource domain are deployed with the same type of resources, and the calculation module 101 is configured to calculate first resource parameters of respective nodes and second resource parameters of respective resource users at the respective nodes in the same resource domain.
In some embodiments, the calculation module 101 is configured to calculate first resource parameters of respective nodes and second resource parameters of respective resource users at the respective nodes according to a preset period; or, calculate first resource parameters of respective nodes and second resource parameters of respective resource users at the respective nodes when at least one of the following conditions is satisfied: a resource user executes on a resource scheduling apparatus at least one of: logging in, logging out, or closing a desktop; a resource user initiates a resource addition request or a resource deletion request; or at least one of the nodes fails and/or hardware of at least one of the nodes fails.
An embodiment of the present disclosure further provides a computer device, including: one or more processors and a storage means. The storage means stores one or more programs thereon which, when executed by the one or more processors, cause the one or more processors to perform the resource scheduling method described in any of the above embodiments.
An embodiment of the present disclosure further provides a computer-readable medium having a computer program stored thereon which, when executed, causes the resource scheduling method described in any of the above embodiments to be implemented.
Those of ordinary skill in the art will appreciate that all or some operations of the above described method, functional modules/units in the apparatus may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or operation may be performed cooperatively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a CPU, a digital signal processor or microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium which may include a computer storage medium (or non-transitory medium) and communication medium (or transitory medium). The term computer storage medium includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. The computer storage medium includes, but is not limited to, an RAM, an ROM, an EEPROM, a flash or any other memory technology, a CD-ROM, a digital versatile disc (DVD) or any other optical disc storage, a magnetic cartridge, a magnetic tape, a magnetic disk storage or any other magnetic storage device, or may be any other medium used for storing the desired information and accessible by a computer. Moreover, it is well known to those ordinary skilled in the art that a communication medium typically includes a computer-readable instruction, a data structure, a program module, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any information delivery medium.
The present disclosure has disclosed exemplary embodiments, and although specific terms are employed, they are used and should be interpreted merely in a generic and descriptive sense, not for purposes of limitation. In some instances, as would be apparent to one skilled in the art, features, characteristics and/or elements described in connection with a particular embodiment may be used alone or in combination with features, characteristics and/or elements described in connection with another embodiment, unless expressly stated otherwise. It will, therefore, be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present disclosure as set forth in the appended claims.
Number | Date | Country | Kind |
---|---|---|---|
202111048559.8 | Sep 2021 | CN | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/CN2022/117806 | 9/8/2022 | WO |