This is a continuation application of International Application PCT/JP2012/073544 filed on Sep. 13, 2012 which claims priority from International Applications PCT/JP2011/071737 filed on Sep. 22, 2011 and PCT/JP2011/073973 filed on Oct. 18, 2011, the contents of which are incorporated herein by reference.
The embodiments discussed herein are related to a computer system and a virtual machine arranging method.
According to a conventional method, the lowest value of energy consumption is calculated using the mixed integer programming for each case in which the models and the number of computers concurrently operated differs; the case having the lowest energy consumption is ultimately selected among the cases; and thereby, plural energy supply units are operated to distribute the load thereto (see, e.g., Japanese Laid-Open Patent Publication No. H6-141468). According to another method, a production request is accepted that includes the number of processors, the memory amount, and an assignment policy for resources, of a virtual server that is to be produced; and processors and memories are assigned to the virtual server based on the accepted production request such that the assignment policy is satisfied (see, e.g., Japanese Laid-Open Patent Publication No. 2009-151745). According to yet another method, power consumption is calculated for each software process based on the amount of resources used when a processor executes the software process; some of the software processes are moved from a first processor to a second processor according to the calculated power consumption (see, e.g., Japanese Laid-Open Patent Publication No. 2010-205200). According to another method, the amount of resources to be assigned to a logical server is calculated for each combination of a logical server and physical server resources to minimize the power consumption of plural servers and that of plural air-conditioners; and the assignment of the resources of the physical server to the logical server is changed (see, e.g., Japanese Laid-Open Patent Publication No. 2011-39889). According to still another method, workloads are assigned to information processing apparatuses using the positions of and operation information concerning the information processing apparatuses, and positions of and environment information concerning power supply facility and cooling facility such that the total power consumption of the information processing apparatuses, the power supply loss of the power supply facility, and electric power for cooling used by the cooling facility is reduced (see, e.g., Japanese Laid-Open Patent Publication No. 2009-252056).
However, in a computer system that includes plural racks respectively having an uninterruptible power supply (UPS) and plural servers, according to the conventional methods, a problem arises in that plural virtual machines cannot be assigned to the plural servers. Even when plural virtual machines are assigned to the plural servers according to one of the conventional methods, another problem arises in that in some cases no electric power saving can be facilitated.
According to an aspect of an embodiment, a computer system includes plural servers in which virtual machines are arranged; plural power supply apparatuses that supply electric power to the servers; and a control apparatus that controls arrangement of the virtual machines in the servers. The control apparatus solves an integer programming problem whose objective function is total power consumption by the servers and by the power supply apparatuses, the total power consumption being described as a function of the arrangement of the virtual machines; and arranges the virtual machines based on a solution of the integer programming problem.
The object and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the claims.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention.
Embodiments of a computer system and a virtual machine arranging method will be described in detail with reference to the accompanying drawings.
The control apparatus 4 controls the arrangement of the virtual machines for the server 2; solves an integer programming problem using, as an objective function, the total power consumption of the servers 2 and the power supply apparatuses 3, described as a function of the virtual machine arrangement; and arranges the virtual machines in the servers 2 based on the solution of the integer programming problem.
According to the first embodiment, a virtual machine arrangement capable of minimizing the total power consumption can be obtained by solving the integer programming problem that uses, as the objective function, the total power consumption of the power consumption of the servers 2 and the power consumption of the power supply apparatuses 3. Therefore, electric energy saving of the computer system 1 can be facilitated.
In a second embodiment, an example will be described of the computer system 1 according to the first embodiment. An example of the computer system 1 can be, for example, a data center. In the data center, many apparatuses such as computers and those for data communication are operated.
In the computer system 1 depicted in
The CPU 11 administers overall control of the control apparatus 4. The CPU 11 solves the integer programming problem and arranges the virtual machines by executing a program that realizes a virtual machine arranging method described hereinafter. The ROM 12 stores programs such as a boot program and the program that implements the virtual machine arranging method. The RAM 13 is used as a work area of the CPU 11.
The HDD 14 controls the reading and writing of data with respect to the hard disk 20. The hard disk 20 stores data written thereto under the control of the hard disk drive 14. The FDD 15 controls the reading and writing of data with respect to the flexible disk 21. The flexible disk 21 stores data written thereto under the control of the FDD 15.
Besides the flexible disk 21, CD-ROM (CD-R, CD-RW), MO, a digital versatile disk (DVD), or a memory card and the like may be used as a removable recording medium. The display 16 displays, for example, data such as text, images, functional information, etc., in addition to a cursor, icons, and/or tool boxes. A cathode ray tube (CRT), a thin-film-transistor (TFT) liquid crystal display, a plasma display, etc., may be employed as the display 16.
The interface 19 is connected to each of the racks 5 via a non-depicted network. The interface 19 controls the input and output of data with respect to the racks 5. A local area network adapter, for example, may be used as the interface 19.
The keyboard 17 includes, for example, keys for inputting letters, numerals, and various instructions and performs the input of data. The keyboard 17 may be a touch-panel-type input pad or numeric keypad, etc. The mouse 18 is used to move the cursor, select a region, or move and change the size of windows. A track ball or a joy stick may be adopted provided each respectively has a function similar to a pointing device.
The control apparatus 4 may be connected to a scanner that takes in images. A printer may be connected to the control apparatus 4, as one example of an output device. The control apparatus 4 may be a computer such as a personal computer and a work station, or an information processing apparatus such as a mobile telephone.
The managing unit 33 rewrites data concerning an initial arrangement of the virtual machines, based on information concerning the virtual machines arranged in the servers 2. The data concerning the initial arrangement includes for each of the servers 2, the number of arranged virtual machines and may be stored in, for example, the storing unit 32.
The storing unit 32 stores data (constants), variables, an objective function, and constraints; inputs the data (the constants), the variables, the objective function, and the constraints into the solving unit 34; and may use memory such as, for example, the ROM 12 or the RAM 13 as the storage medium thereof. The data (the constants), the variables, the objective function, and the constraints that do not dynamically vary may be described in the program that realizes the virtual machine arranging method.
The determining unit 31 determines the integer programming problem that is to be solved among integer programming problems that correspond to the initial arrangement of the virtual machines, an additional arrangement thereof, and a rearrangement thereof. The determining unit 31 issues an instruction to the solving unit 34 based on the result of the determination thereof.
Solving the integer programming problem corresponding to the initial arrangement of the virtual machines enables an arrangement of the one or more virtual machine(s) in one or more server(s) 2 in a state where all the servers 2 discontinue operation. The initial arrangement of the virtual machines in this case is the state where no virtual machine is arranged in any of the servers 2.
Solving the integer programming problem corresponding to the additional arrangement of the virtual machines enables an arrangement of further one or more virtual machine(s) to further one or more server(s) 2 in addition to the initial arrangement state where the one or more virtual machine(s) is/are already arranged in the one or more server(s) 2. However, none of the virtual machines arranged in the initial arrangement state is moved to any other server 2.
Solving the integer programming problem corresponding to the rearrangement of the virtual machines enables a rearrangement of the virtual machines using live migration for the initial arrangement state where the one or more virtual machine(s) is/are already arranged in the one or more server(s) 2. Without discontinuing the operation of one or more virtual machine(s) arranged in a server 2, the virtual machine(s) can be arranged in another server 2 by using the live migration. The control apparatus 4 may monitor the operation time period of the data center using a timer, etc. included therein and may execute the rearrangement of the virtual machines each time a specific time period elapses.
The solving unit 34 solves the integer programming problems based on the data (the constants), the variables, the objective function, and the constraints delivered from the storing unit 32 and the instruction from the determining unit 31; may solve an integer programming problem (formulated by, for example, Eqs. (8) to (12) described later) when the initial arrangement of the virtual machines is executed; may solve an integer programming problem (formulated by, for example, Eqs. (14) to (18) described later) when the additional arrangement of the virtual machines is executed; and may solve an integer programming problem (formulated by, for example, Eqs. (36) to (50) described later) when the rearrangement of the virtual machines is executed.
An example of the solving unit 34 can be, for example, software to solve the integer programming problems (a solver). Examples of the solver can be, for example, “GLPK”, “SYMPHONY”, and “Gurobi Optimizer”. The arranging unit 35 arranges the virtual machines in the servers 2 based on the solution of the integer programming problem derived by the solving unit 34.
For example, “c” may be a value obtained by dividing the average operation time period of the live migration by the average operation time period of the virtual machines after the rearrangement. “c” in this case is a value less than one. Selecting c in this manner enables minimization of the total power consumption of the power consumption of the servers 2 and that of the power supply apparatuses 3.
The values of N and S are determined in advance based on the configuration of the data center. The values of c, Lij, αij, βij, γ−ij, and γ+ij are determined in advance based on the server. The values of η1 and εi are determined in advance based on the power supply apparatus. “wij[0]” is dynamically rewritten by the managing unit 33 during the operation of the data center. M is determined according to the number of applications to be started up, etc.
The storing unit 32 inputs the data (the constants), the variables, the objective function, and the constraints into the solving unit 34 (step S2). The solving unit 34 solves the integer programming problem using, for example, the solver based on the data (the constants), the variables, the objective function, and the constraints that are input thereinto, and the instruction from the determining unit 31 to obtain a solution (step S3). The formulation as the integer programming problem will be described later for each type of arrangement. The arranging unit 35 arranges the virtual machines based on the solution obtained by the solving unit 34 (step S4). The series of process steps of the virtual machine arrangement process come to an end.
Modeling will be described for the power consumption of the server, the power consumption of the power supply apparatus, the total power consumption of the servers and the power supply apparatus in the rack, and the total power consumption of the servers and the power supply apparatuses in the data center.
The server has the properties of (S-1) to (S-3) below concerning the power consumption thereof.
(S-1) When at least one virtual machine is executed, the power of the server is turned on. When the number of execution sessions of the virtual machine is zero, the power of the server is turned off.
(S-2) When the power is turned on, a specific amount of electric power is consumed.
(S-3) When the number of executed virtual machines is increased, the power consumption is increased according to the increase.
A variable representing the number of virtual machines executed by the j-th server in the i-th rack is represented by “wij”. “wij” is an integer variable greater than or equal to zero and less than or equal to Lij. “vij” is a variable that takes the value of one or zero for the j-th server in the i-th rack. A constraint below is imposed on vij.
v
ij
≦w
ij
≦L
ij
v
ij (1)
In Eq. (1), in a case where wij is greater than or equal to one, when vij is zero, the inequality on the right side of Eq. (1) does not hold and therefore, vij is one. On the other hand, in a case where wij is zero, when vij is one, the inequality on the left side of Eq. (1) does not hold and therefore, vij is zero.
Therefore, the value of vij indicates the turning on or off of the power of the server. When the value of vij is zero, this indicates that the power of the server is turned off and, when the value of vij is one, this indicates that the power thereof is turned on. Eqs. (2) and (3) express the above.
w
ij≧1server power ONvij=1 (2)
w
ij=0server power OFFvij=0 (3)
The power consumption of the j-th server in the i-th rack is expressed by Eq. (4) below using the constants and variables. Eq. (4) expresses the properties of (S-1) to (S-3) using linear expressions of the variables wij and vij.
βijvij+αijwij (4)
In Eq. (4): “αijwij” corresponds to [the electric power consumed per one virtual machinexthe number of virtual machines] and therefore, represents the electric power consumed corresponding to the number of virtual machines; and “βijvij” becomes βij when the power of the server is turned on, becomes zero when the power of the server is turned off and therefore, represents the electric power consumed by the turning on of the power of the server.
The power supply apparatus has properties of (U-1) and (U-2) below concerning the power consumption thereof.
(U-1) A specific amount of electric power is continually consumed regardless of the operation state of the servers in the rack.
(U-2) When the total power consumption of the servers in the rack is increased, the power consumption is increased according to the increase.
The power consumption of the power supply apparatus equipped to the i-th rack is expressed by Eq. (5) below based on the properties of (U-1) and (U-2). Eq. (5) expresses the properties of (U-1) and (U-2) using a linear expression of the variables wij and vij.
In Eq. (5), [Σ(βijvij+αijwij)] (Σ index “j” is omitted) represents the total power consumption of the servers in the i-th rack. The second term of Eq. (5) obtained by multiplying the total power consumption of the servers in the i-th rack by εi represents the electric power consumed by the power supply apparatus corresponding to the total power consumption of the servers in the i-th rack. “ηi” represents the electric power continually consumed by the power supply apparatus of the i-th rack.
As described in the section for the modeling of the power consumption of the power supply apparatus, the total power consumption of the servers in the i-th rack is [Σ(βijvij+αijwij)] (Σ index “j” is omitted). The power consumption of the power supply apparatus equipped in the i-th rack is expressed by Eq. (5). The power consumption obtained by adding these is the total power consumption of the i-th rack. Therefore, the total power consumption of the i-th rack is expressed by Eq. (6) below.
The number of servers in the rack may be variable. The number of servers in the rack being variable can be represented by a constraint of [wij=0, j≧s], for solving the integer programming problem representing the number of servers in the i-th rack as “s”, which is an integer smaller than the integer S.
The sum of Eq. (6) for the first to the N-th racks gives the total power consumption in the data center. Therefore, the total power consumption in the data center is expressed by Eq. (7) below.
Formulation as an integer programming problem will be described for the initial arrangement, the additional arrangement, and the rearrangement of the virtual machines.
When the initial arrangement of the virtual machines is executed, no server has any virtual machine arranged therein. Therefore, the power of all the servers is turned off.
The electric power saving problem of the servers and the power supply apparatuses arising when the initial arrangement of the virtual machines is executed can be formulated as an integer programming problem expressed by Eqs. (8) to (12) below. Eq. (8) expresses the objective function. Eqs. (9) to (12) express the constraints. Eq. (11) expresses the constraints for the total number of the virtual machines to be arranged to be equal to the number of virtual machines that needs to be arranged. The decision variables are wij, vij, i=1, . . . , and N, and j=1, . . . , and S.
It is assumed as the precondition for executing the additional arrangement of the virtual machines that wij[0] virtual machines are already arranged in the j-th server in the i-th rack. It is also assumed that the currently arranged virtual machines are not migrated. Therefore, virtual machines are additionally arranged in the servers already having wij[0] virtual machines arranged therein as the initial state and therefore, wij is a value greater than or equal to wij[0]. Expressing this as a constraint gives Eq. (13) below.
w
ij
ε{w
ij[0], . . . ,Lij},i=1, . . . ,N,j=1, . . . ,S (13)
The number of additionally arranged virtual machines is represented by “ΔM”. The number of virtual machines arranged in the initial arrangement state is [ΣΣwij[0]] (former Σ index “i” and latter Σ index “j” are omitted) and therefore, the number of virtual machines after the additional arrangement is [ΣΣwij[0]+ΔM] (former Σ index “i” and latter Σ index “j” are omitted).
Therefore, the electric power saving problem of the servers and the power supply apparatus arising when the additional arrangement of the virtual machines is executed can be formulated as an integer programming problem expressed by Eqs. (14) to (18) below. Eq. (14) expresses the objective function. Eqs. (15) to (18) express the constraints. Eq. (17) expresses the constraints for the total number of the virtual machines after the additional arrangement to be equal to the value obtained by adding the number of virtual machines in the initial arrangement state and the number of additionally arranged virtual machines. The decision variables are wij, vij, i=1, . . . , and N, and j=1, . . . , and S.
It is assumed as the precondition for executing the rearrangement of the virtual machines that wij[0] virtual machines are already arranged in the j-th server in the i-th rack. In a rearrangement of the virtual machines, the virtual machines are migrated using the live migration. When a virtual machine is migrated using the live migration, both the migration source server and the migration destination server consume electric power.
“δ−ij” is a variable taking a value of zero or one for the j-th server in the i-th rack. A constraint below is imposed on δ−ij. Here, “d” is a positive real number smaller than one.
d+(−Lij−d)δ−ij≦wij−wij[0]≦Lij(1−δ−ij) (19)
In Eq. (19), in a case where [wij−wij 0]] is less than or equal to zero, when δ−ij is zero, the inequality on the left side of Eq. (19) does not hold and therefore, δ−ij is one. On the other hand, in a case where [wij−wij[0]] is greater than or equal to one, when δ−iJ is one, the inequality on the right side of Eq. (19) does not hold and therefore, is zero. Eqs. (20) and (21) express the above.
w
ij
−w
ij[0]≦0δ−ij=1 (20)
w
ij
−w
ij[0]≧1δ−ij=0 (21)
“z−ij” is an integer variable taking a value that is any one of zero to L for the j-th server in the i-th rack. Constraints below are imposed on z−ij.
−Lijδij−≦zij−≦Lijδij− (22)
−(wij−wij[0])−Lij(1−δij−)≦zij−≦−(wij−wij[0])+Lij(1−δij−) (23)
When [wij−wij[0]] is greater than or equal to one, δ−ij is zero based on Eq. (21). Therefore, z−ij is zero. When wij−wij[0], δ−ij, and z−ij are [wij−wij [0]≧1, δ−ij=0, and z−ij=0], [−(wij−wij[0])+Lij≧0≧−(wij−wij[0])−Lij] in Eq. (23). Therefore, Eq. (23) holds. Eq. (24) below expresses this.
w
ij
−w
ij[0]≧1zij−=0 (24)
On the other hand, when [wij−wij[0]] is less than or equal to zero, δ−ij is one based on Eq. (20). Therefore, Eq. (23) is [−(wij−wij[0])≦z−ij−(wij−wij[0])]. Therefore, z−ij is equal to [−(wij−wij[0])].
When wij−wij[0], δ−ij, and z−ij are [wij−wij[0]≦0, δ−ij=1, and zij−=−(wij−wij [0])], [−Lij≦−(wij−wij [0])≦Lij] in Eq. (22). Therefore, Eq. (22) holds. Eq. (25) below expresses this.
w
ij
−w
ij[0]≦0zij−=−(wij−wij[0]) (25)
[wij−wij[0]] being less than or equal to zero means that the virtual machines are migrated from the j-th server in the i-th rack to another server. [wij−wij[0]] being greater than or equal to one means that the virtual machines are migrated to the j-th server in the i-th rack from another server. “zij−” represents the number of virtual machines that migrate from the j-th server in the i-th rack to another server.
Therefore, the electric power consumed when the z−ij virtual machines are migrated from the j-th server in the i-th rack to the other server is expressed by Eq. (26) below.
γij−zij− (26)
“δ+ij” is a variable taking a value of zero or one for the j-th server in the i-th rack. A constraint below is imposed on δ+ij.
δij+=1−δij− (27)
Eqs. (28) and (29) are obtained from Eqs. (20) and (21) that are the constraints of Eq. (27).
w
ij
−w
ij[0]≦0δij+=0 (28)
w
ij
−w
ij[0]≧1δij+=1 (29)
“z+ij” is an integer variable taking a value that is any one of zero to Lij for the j-th server in the i-th rack. Constraints below are imposed on z+ij.
−Lijδij+≦zij+≦Lijδij+ (30)
(wij−wij[0])−Lij(1−δij+)≦zij+≦(wij−wij[0])+Lij(1−δij+) (31)
When [wij−wij[0]] is greater than or equal to one, δ+ij is one from Eq. (29). Therefore, Eq. (31) is [wij−wij[0]≦z+ijwij−wij[0]] and therefore, z+ij is equal to [wij−wij[0]].
When [wij−wij[0]], δ+ij, and z+ij are [wij−wij[0]]≧1, δ+ij=1, and z+ij=wij−wij[0], [−Lij≦wij−wij[0]≦Lij] is obtained in Eq. (30). Therefore, Eq. (30) holds. Eq. (32) expresses this.
w
ij
−w
ij[0]≧1zij+=wij−wij[0] (32)
On the other hand, when [wij−wij[0]] is less than or equal to zero, δ+ij is zero from Eq. (28). Therefore, Eq. (30) is [0≦z+i,j≦0] and therefore, z+ij is zero. When [wij−wij[0]], δ+ij, and z+ij are [wij−wij[0]]≦0, δ+ij=0, and z+ij=0, [(wij−wij[0])−Lij≦0≦(wij−wij[0])+Lij] is obtained in Eq. (31). Therefore, Eq. (31) holds. Eq. (33) expresses this.
w
ij
−w
ij[0]≦0zij+=0 (33)
“z+ij” represents the number of virtual machines that are migrated to the j-th server in the i-th rack from another server. Therefore, the power consumption consumed when z+ij virtual machines are migrated to the j-th server in the i-th rack from another server is expressed by Eq. (34) below.
γij+zij+ (34)
The electric power consumed by the live migration is obtained by totaling, for the j-th server in the i-th rack, the power consumption consumed when the z−ij virtual machines are migrated therefrom to another server and the power consumption consumed when the z+ij virtual machines migrate thereto from another server, and further totaling this total for all the servers. Therefore, the electric power consumed by the live migration is expressed by Eq. (35) below.
Therefore, the electric power saving problem of the servers and the power supply apparatuses arising when the rearrangement of the virtual machines is executed can be formulated as an integer programming problem expressed by Eqs. (36) to (50) below. Eq. (36) expresses the objective function. Eqs. (37) to (50) express the constraints. The decision variables are wij, vij, δ−ij, δ+ij, zij−, z+ij, i=1, . . . , and N, and j=1, . . . , and S.
According to the second embodiment, the same effect as that of the first embodiment is achieved for the initial arrangement, the additional arrangement, and the rearrangement of the virtual machines, respectively.
In a third embodiment, a case will be described as an example of the computer system 1 according to the first embodiment, where, for example, an air conditioning apparatus is installed in the data center of the second embodiment and the electric power saving is facilitated for the overall data center including the power consumption of the air conditioning apparatus. In the third embodiment, the size of the virtual machines may differ from one another. Components identical to those in the second embodiment are given the same reference numerals used in the second embodiment and will not again be described.
In the depicted example, two CRAC units 7 and two chiller plants 8 are depicted. However, the number of CRAC units 7 and chiller plants 8 may each be one, or three or more. Plural CRAC units 7 may be connected to one chiller plant 8. In this example, description will be made assuming that one CRAC unit 7 and one chiller plant 8 form a pair.
The solving unit 34 solves the mixed integer programming problems (described hereinafter), based on the data (the constants), the variables, the objective function, and the constraints delivered from the storing unit 32. An example of the solving unit 34 can be, for example, software to solve the mixed integer programming problems (a solver). Examples of the solver can be, for example, “GLPK”, “SYMPHONY”, and “Gurobi Optimizer”. Similar to the second embodiment, when the initial arrangement, the additional arrangement, and the rearrangement of the virtual machines are all coped with, the control apparatus 4 may be configured to include the determining unit 31 as depicted in
The computer system 1 may have the data N and S defining the configuration of the data center (
The values of C, λk, μk, and κk are determined in advance based on the configuration of the air conditioning apparatus. The values of φki and qsup,i are determined in advance based on the configurations of the rack and the air conditioning apparatus. “w0ij” is dynamically rewritten by the managing unit 33 during the operation of the data center.
Formulation as a mixed integer programming problem and modeling of virtual machines and arrangement thereof will be described. As depicted in
Modeling of the power consumption of the server, the power consumption of the power supply apparatus, the power consumption of the CRAC unit, the power consumption of the chiller plant, and the power consumption of the overall data center will be described.
The power consumption of the server can be modeled similarly to that of the second embodiment. In the third embodiment, the server has the properties of (S-1) and (S-2) described in the second embodiment and the property of (S-4) below.
(S-4) When the total size of the executed virtual machines is increased, the load increases and, corresponding to this, the power consumption increases.
The power consumption that is consumed when the virtual machines whose total size is w0ij are initially arranged in the j-th server in the i-th rack and a virtual machine whose size is τn is further arranged in this server is expressed by Eq. (51) below. Eq. (51) expresses the properties of (S-1), (S-2), and (S-4) using linear expressions of the variables xn,i,j and vij. “vij” is same as that described in the second embodiment.
The power consumption of the power supply apparatus can be modeled similarly to that of the second embodiment. In the third embodiment, the power supply apparatus has the properties of (U-1) and (U-2) described in the second embodiment. The power consumption of the server is expressed by Eq. (51) and therefore, the power consumption of the power supply apparatus is expressed by Eq. (52) below. Eq. (52) expresses the properties of (U-1) and (U-2) using the linear expressions of the variables xn,i,j and vij.
In modeling the power consumption of the CRAC unit, the CRAC unit has the properties of (C-1) and (C-2) concerning the power consumption thereof.
(C-1) A specific amount of electric power is always consumed.
(C-2) When the heat flow entering the k-th CRAC unit is increased, the power consumption of the k-th CRAC unit is increased.
The power consumption of the k-th CRAC unit is expressed by Eq. (53) below based on the properties of (C-1) and (C-2). In Eq. (53), “qin, CRAC, k” represents the heat flow (unit: W) that enters the k-th CRAC unit.
λk+μkqin,CRAC,k (53)
In the modeling of power consumption of the chiller plant, the chiller plant has the properties of (C-3) and (C-4) below concerning power consumption.
(C-3) The heat flow entering the k-th chiller plant is equal to the heat flow discharged from the k-th CRAC unit.
(C-4) When the heat flow discharged from the k-th chiller plant increases, the power consumption of the k-th chiller plant increases.
Eq. (54) below expresses the power consumption of the k-th chiller plant based on the properties of (C-3) and (C-4). In Eq. (54), qout, CRAC, k represents the heat flow (unit: W) flowing out from the k-th CRAC unit. When the chiller plant continually consumes electric power, similarly to the CRAC unit, a term for the base power consumption of the chiller plant is added to Eq. (54).
κkqout,CRAC,k (54)
The total power consumption of the servers and the power supply apparatuses in the data center is the power consumption obtained by totaling Eq. (51) for the first to the S-th servers and adding Eq. (52) to this total; this addition result for all of the first to the N-th racks. The power consumption of the overall data center is the power consumption obtained by totaling: the total power consumption of the servers and the power supply apparatuses; Eq. (53) for all of the first to the C-th CRAC units; and Eq. (54) for all of the first to the C-th chiller plants. Therefore, the power consumption of the overall data center is expressed by Eq. (55) below.
Modeling of the heat flows will be described. For the heat flows, the relations of (Q-1) to (Q-3) below hold between the racks and the air conditioning apparatus.
(Q-1) The heat flow discharged from each of the CRAC units is obtained by totaling the heat flow entering the CRAC unit and the heat flow generated by the CRAC unit (that is, the power consumption thereof).
(Q-2) The heat flow entering the CRAC units is the heat flow discharged from the racks.
(Q-3) The heat flow discharged from the racks is calculated by totaling the heat flow entering the racks and the heat flow generated by the racks (that is, the power consumption thereof).
Based on the relation of (Q-1), the heat flow discharged from the k-th CRAC unit is expressed by Eq. (56) below.
q
out,CRAC,k
=q
in,CRAC,k+λk+μkqin,CRAC,k (56)
Based on the relation of (Q-2), the heat flow entering the k-th CRAC unit is expressed by Eq. (57) below. In Eq. (57), “qout, RACK, i” is a variable representing the heat flow (unit: [W]) flowing out from the i-th rack.
Based on the relation of (Q-3), the heat flow discharged from the i-th rack is expressed by Eq. (58) below.
From Eq. (58), the heat flow qout,RACK,i discharged from the i-th rack is a function of the variables xn,i,j and vij. Therefore, from Eq. (57), the heat flow qin,CRAC,k entering the k-th CRAC unit is also a function of the variables xn,i,j and vij. From Eq. (56), the heat flow qout,CRAC,k discharged from the k-th CRAC unit is also a function of the variables xn,i,j and vij.
From the above, the electric power saving problem of the total power consumption of the servers, the power supply apparatuses, the CRAC units, and the chiller plants consumed when the virtual machines are arranged can be formulated as a mixed integer programming problem expressed by Eqs. (59) to (66) below. Eq. (59) expresses the objective function. Eqs. (60) to (66) express the constraints. The decision variables are xn,i,j, vij, gin,CRAC,k, qout,CRAC,k, qout,RACK,i, n=1, . . . , and M, i=1, . . . , and N, j=1, . . . , and S, and k=1, . . . , and C.
According to the third embodiment, the mixed integer programming problem is solved that uses, as the objective functions thereof, the total power consumption of the power consumption of the servers 2, the total power consumption of the power supply apparatuses 3, the total power consumption of the CRAC units 7, and the total power consumption of the chiller plants 8; and thereby, an arrangement of the virtual machines can be obtained that can minimize the total power consumption. Therefore, the electric power saving of the computer system 1 can be facilitated. Even when the size of virtual machines differs from one another, an arrangement of the virtual machines is obtained that can minimize the total power consumption and thereby, the electric power saving of the computer system 1 can be facilitated.
According to the third embodiment, the power consumption of the CRAC units 7 and that of the chiller plants 8 are each expressed by a linear expression and the mixed integer programming problem is solved; and thereby, the solution can be obtained more quickly than in a case where the power consumption of the air conditioning apparatus is obtained by thermal fluid simulation or heat exchange simulation. Consequently, the data center 6 can be smoothly operated.
According to the third embodiment, a solution can be obtained that is more suitable for the actual state of the operation of the data center 6 compared to a case where the air conditioning equipment is virtually disassembled to match with the number of servers 2 and a solution is obtained that can minimize the power consumption of the data center 6.
The size τn may be set to be same for all the virtual machines to be arranged. With such setting, the size becomes the same for all the virtual machines to be arranged, similarly to the second embodiment and therefore, according to the third embodiment, the effect of the electric power saving can also be achieved even when the size of all the virtual machines to be arranged is the same.
The total size w0ij of the initially arranged virtual machines in all the servers 2 may be set to be zero. With such setting, in the initial state, no virtual machines are arranged in the servers 2, similar to the initial arrangement of the virtual machines in the second embodiment. Therefore, according to the third embodiment, the effect of the electric power saving can also be achieved even when the virtual machines are initially arranged.
As to the objective function of Eq. (59) and the constraints of Eqs. (64) to (66), the integer programming problem may be solved without the terms and expressions concerning the heat flows of the air conditioning apparatus. Thus, electric power saving can be facilitated in a case where virtual machines of different sizes are arranged, without taking into consideration the power consumption of the air conditioning apparatus.
A fourth embodiment is an example where, in the computer system 1 according to the third embodiment, the M virtual machines to be arranged are arranged in a single rack 5 as much as possible. In a case where, for example, the M virtual machines are included in one service, when the service executes, for example, communication, an arrangement of the M virtual machines in the same rack 5 may enable an increase of the communication speed. Components identical to those in the third embodiment are given the same reference numerals used in the third embodiment and will not again be described.
In the fourth embodiment, as expressed by Eq. (67), for the i-th rack, “σi” is an auxiliary variable taking a value of zero or one.
σiε{0,1},i=1, . . . ,N (67)
A constraint expressed by Eq. (68) is imposed on σi.
In a case where Σxn,i,j (Σ indexes n and j are omitted) is greater than or equal to one in Eq. (68), when σi is zero, the inequality on the right side of Eq. (68) does not hold. Therefore, σi is one. On the other hand, in a case where Σxn,i,j (Σ indexes n and j are omitted) is zero, when σi is one, the inequality on the left side of Eq. (68) does not hold. Therefore, σi is zero.
Therefore, the value of σi indicates whether the virtual machines are executed in the i-th rack. The value of σi is one when the virtual machines are executed in the i-th rack, and is zero in other cases, that is, when the virtual machines are not executed therein. Eqs. (69) and (70) express this. The total of the values of σi for the first to the N-th racks is the number of racks for the newly arranged virtual machines to be executed.
From the above, the electric power saving problem for a case where the virtual machines are arranged preferentially in the same rack can be formulated as a mixed integer programming problem expressed by Eqs. (71) to (80) below. Eq. (71) expresses the objective function. Eqs. (72) to (80) express the constraints. The decision variables are xn,i,j, Vij, σi, qin,CRAC,k, qout,CRAC,k, qout,RACK,i, n=1, . . . , and M, i=1, . . . , and N, j=1, . . . , and S, and k=1, . . . , and C.
According to the fourth embodiment, the same effect as that of the third embodiment is achieved, and the arrangement of the plural virtual machines in the same rack enables improvement of the communication speed and of the electric power saving to be expected.
A fifth embodiment is an example where the virtual machines are rearranged when the size of the virtual machines differs from one another. For example, the fifth embodiment is an example where the electric power consumed by the servers, the UPSs, and the air-conditioner is modeled as a function of the arrangement of the virtual machines of different sizes. A computer system according to the fifth embodiment formulates an electric power saving problem as a mixed integer programming problem based on the virtual machine rearrangement using the virtual machine migration, obtains the solution thereof and thereby, calculates the arrangement of the virtual machines to achieve electric power saving.
Thereby, even when the size of the virtual machines differs from one another, virtual machine rearrangement that is optimal for the electric power saving can be executed. In the fifth embodiment, electric power saving will be described for the computer system depicted in
In the fifth embodiment, the computer system 1 may have the data N and S defining the configuration of the data center (
In the fifth embodiment, the computer system 1 may have the data defining the number of air conditioning apparatuses (
Formulation of the mixed integer programming problem according to the fifth embodiment will be described. In the fifth embodiment, similar to the third embodiment, the power consumption of the CRAC unit and that of the chiller plant are also modeled. However, the modeling of the power consumption of the CRAC unit and that of the chiller plant are same as those in the third embodiment and therefore, will not again be described.
Modeling of a virtual machine and virtual machine arrangement will be described. The service count is “V” as depicted in
“xm,n,i,j” is a variable that takes a value of “one” when the n-th virtual machine of the m-th service is executed by the j-th server in the i-th rack, and takes a value of “zero” when the n-th virtual machine is not executed thereby. “wij” is a variable representing the total size of the virtual machines executed by the j-th server in the i-th rack. In this case, xm,n,i,j and wij are related to one another as expressed by Eq. (81).
w
ij=Σm,nτmnxm,n,i,j (81)
Modeling of power consumption of a server will be described. The execution of at least one virtual machine by the j-th server in the i-th rack is indicative of the power of the server already being turned on. Therefore, the turning on of the power consumes some electric power. When the total size of the virtual machines executed by the j-th server in the i-th rack increases, the load increases and more electric power is consumed. Therefore, the power consumption of the j-th server in the i-th rack is modeled as below.
βijvij+αijΣm=1VΣn=1Mτmnxm,n,i,i (82)
In Eq. (82), “vij” is a variable representing the turning on or off of the j-th server in the i-th rack, takes a value of “one” for the turning on, and a value of “zero” for the turning off. “βij” represents the base power consumption consumed by the j-th server alone when the j-th server in the i-th rack is turned on, and is a constant whose unit is [W]. “αij” represents the electric power consumed per unit size (one core or one thread) by the j-th server in the i-th rack, and is also a constant whose unit is [W (watt)].
Modeling of power consumption of the UPS will be described. The UPS continually consumes a specific amount of electric power. In addition, electric power is consumed of an amount corresponding to the power consumption of the servers mounted on the rack to which the UPS is equipped. Therefore, the power consumption of the UPS in the i-th rack is modeled as below.
ηi+εiΣj=1S(βijvij+αijΣm,nτmnxm,n,i,j) (83)
In Eq. (83), “S” represents the number of servers per one rack, and is a constant. “ηi” represents the base power consumption continually consumed by the UPS in the i-th rack, and is a constant whose unit is [W (watt)] (see
Items for the overall data center are modeled such as the electric power consumed by the servers, the UPSs, the CRAC units, and the chiller plants, respectively.
In Eq. (84): “N” represents the number of racks in the data center; “C” represents the respective number of the CRAC units and the chiller plants in the data center.
Based on the relation of (C-1) described in the third embodiment, the heat flow discharged from the k-th CRAC unit is as expressed by Eq. (56). Based on the relation of (Q-2) described in the third embodiment, the heat flow entering the k-th CRAC unit is as expressed by Eq. (57). Based on the relation of (Q-3) described in the third embodiment, the heat flow discharged from the i-th rack is expressed by Eq. (85) below.
In Eq. (85), “qsup,i” represents the heat flow supplied from the CRAC unit to the i-th rack, and is a constant whose unit is [W (watt)].
Modeling of migration cost will be described. When the migration of the virtual machines is executed, the migration source and the migration destination of the virtual machine consume some electric power. Therefore, the power consumption of the migration of the virtual machines is modeled as below.
Σi=1NΣj=1Sγ−ijzij−+Σi=1NΣj=1Sγ+ijz+ij (86)
zij−: An integer variable greater than or equal to zero and less than or equal to Lij, whereby wij−wji0≧1→zij−=0, and wij−wji0≦0→zij−=−(wij−wji0)
zij+: An integer variable greater than or equal to zero and less than or equal to Lij, whereby wij−wji0≧0→zij+=wij−wji0, and wij−wji0≦0→zij+=0
wij: An integer variable greater than or equal to zero and less than or equal to Lij and represents the total size of the virtual machines executed by the server (i, j)
wij0: The total size (a constant) of the virtual machines already arranged in the server (i, j)
γij−: The electric power consumed per unit size when the virtual machine leaves the j-th server in the i-th rack=(a constant, unit: [W (watt)])
γij1: The electric power consumed per unit size when one virtual machine enters from the j-th server in the i-th rack=(a constant, unit: [W (watt)])
Thus, the electric power minimization problem of the servers, the UPSs, the CRAC units, and the chiller plants for the rearrangement of the virtual machines can be formulated as a mixed integer programming problem below.
The mixed integer programming problem merely has to be solved such that the value of Eq. (87) is minimized. A proper method of determining the scaling factor c (>0) may be an equation “c=the average time period of the migration of the virtual machines/the average operation time period of the virtual machines after the rearrangement”. Eq. (87) complies with the conditions expressed by Eqs. (88) to (107) below.
In Eq. (88), “xm,n,i,j” represents a variable that takes a value of zero or one. The value of xm,n,i,j takes a value of “one” when the n-th virtual machine in the m-th service is executed (arranged) by the j-th server in the i-th rack, and takes a value of “zero” in other cases. “xm,n,i,j” represents the arrangement of the virtual machine.
In Eq. (93), δ−ij is a variable that takes a value of zero or one. The constraint expressed by Eq. (99) is imposed on δ−ij. “d” is a positive real number smaller than one. Based on the constraint expressed by Eq. (99), δ−ij satisfies the relations below.
w
ij
−w
ij
0≦0δij−=1 (108)
w
ij
−w
ij
0≧1δij−=0 (109)
“δ−iJ” has been described in the above order. However, conversely, “a variable that takes a value of zero or one and that satisfies Eqs. (108) and (109)” is necessary and therefore, Eq. (99) is imposed on the variable δ−ij as a constraint to cause δ−ij to have the above property.
In Eqs. (110) and (111), is a variable that takes a value of zero or one as expressed by Eq. (94). The constraint expressed by Eq. (102) is imposed on δ+ij. Based on the constraint expressed by Eq. (102), δ+ij satisfies the relations below.
w
ij
−w
ij
0≦0δij+=0 (110)
w
ij−wij0≧1δij+=1 (111)
In Eqs. (112) and (113), “vij” is a variable that takes a value of zero or one as expressed in Eq. (90). The constraint expressed by Eq. (98) is imposed on vij. Based on the constraint expressed in Eq. (98), vij satisfies the relations below.
“Σm,nτmnxm,n,i,j” represents the total size of the virtual machines arranged in the j-th server in the i-th rack and therefore, “Σm,n,τmnxm,n,i,j=0” represents that no virtual machine is arranged in the j-th server in the i-th rack. No virtual machine being arranged in the j-th server in the i-th rack is indicated by vij being “zero”, and one or more virtual machine(s) being arranged in the j-th server in the i-th rack is indicated by vij being “one”. In the fifth embodiment, it is assumed that the power of the server is turned off to reduce power consumption when no virtual machine is arranged in the server. Therefore, “vij=1” represents that the power of the j-th server in the i-th rack is turned on, and “vij=0” represents that the power of the j-th server in the i-th rack is turned off.
As described, the mixed integer programming problem formulated in the fifth embodiment is solved by and the solution thereof is obtained by the solver such as GLPK, SYMPHONY, or Gurobi Optimizer. Even when the sizes of the virtual machines are different from each other, the control apparatus of the computer system executes the migration of the virtual machines according to the solution obtained by the solver and thereby, the electric power saving of the data center can be facilitated.
A sixth embodiment is an example where virtual machines in the same service are arranged preferentially in the servers in the same rack. In the computer system, the servers are each connected to a network switch and each communicates with other servers through the network switch. Therefore, the communication delay among the servers connected to the same network switch is shorter than that for communication among the servers connected to different network switches. Similarly, when the servers each having a virtual machine arranged therein are connected to the same network switch, the delay of the communication among the virtual machines is also shorter than that among the virtual machines arranged in servers connected to different network switches.
Therefore, preferably, the virtual machines are arranged in servers connected to the same network switch, to reduce the communication delay among the virtual machines constituting one service. In the sixth embodiment, a state is assumed that one network switch is equipped to each of the racks and the servers in the rack are connected to the one network switch. Therefore, the virtual machines being arranged in the servers connected to the same network switch is indicative of the virtual machines being arranged in servers in the same rack. An example will be described where auxiliary variables are introduced, the objective function is changed and thereby, the electric power saving is achieved and the virtual machines included in the same service are arranged in one rack as much as possible.
In the sixth embodiment, as an example of the computer system 1 according to the first embodiment, electric power saving will be described of the computer system depicted in
In the sixth embodiment, the computer system 1 has the data N and S defining the configuration of the data center (
In the sixth embodiment, the computer system 1 may have the data defining the size of the virtual machines (
To add to the fifth embodiment, the computer system 1 of the sixth embodiment may have the data defining the number of services (
Formulation of the mixed integer programming problem according to the sixth embodiment will be described. Contents identical to that of the modeling described in each of the first to the fifth embodiments will not again be described.
Modeling of preferential arrangement to the same rack will be described. The used-rack count penalty is expressed by Eq. (114).
αΣi=1Nσm,i (114)
σm,i:σm,i=0Σn,jzm,n,i,j=0(115)
σm,i=1Σn,jxm,n,i,j≧1 (116)
Eq. (115) expresses that none of the virtual machines of the m-th service is arranged in the i-th rack. Eq. (116) expresses that one or more of the virtual machines of the m-th service is/are arranged in the i-th rack. “Σσm,i” in Eq. (114) represents the number of racks for the virtual machines included in the m-th service to be executed thereby. “a” is the used-rack count penalty coefficient. The used-rack count penalty coefficient a is the penalty coefficient per one rack, and is a constant whose unit is [W (watt)].
Based on the above, the electric power minimization problem of the servers, the UPSs, the CRAC units, and the chiller plants by the rearrangement of the virtual machines can be formulated as a mixed integer programming problem below. Eq. (117) complies with the condition expressed by Eq. (108) in addition to those expressed by Eqs. (88) to (107).
The mixed integer programming problem merely has to be solved such that the value of Eq. (117) is minimized. Eq. (117) is an equation formed by adding Eq. (114) to Eq. (87) described in the fifth embodiment. When the virtual machines constituting the same service are distributed among plural racks, the value of the last term of Eq. (117) increases. On the other hand, when the virtual machines are gathered in the same rack, the value of the term decreases. The rack-use penalty may be applied to Eq. (36) of the second embodiment, Eq. (59) of the third embodiment, and Eq. (71) of the fourth embodiment.
As described, the mixed integer programming problem formulated in the sixth embodiment is solved by, for example, the solver such as GLPK, SYMPHONY, or Gurobi Optimizer. The control apparatus of the computer system executes the migration of the virtual machines according to the solution obtained by the solver and rearranges the virtual machines. Thereby, the electric power saving of the data center can be facilitated considering that the virtual machines included in the same service are arranged in the same rack as much as possible.
A seventh embodiment is an example of electric power saving that takes into consideration the startup stand-by time period, the virtual machine down time, and the migration stand-by time period. The “startup stand-by time period” is a stand-by time period caused by the starting up of the server. A specific time period is necessary for the virtual machines to be arranged after the turning on of the power of the server, and the startup stand-by time period is the stand-by time period generated thereby. The “virtual machine down time” is a time period generated by a discontinuation of the operation of the virtual machines. When the migration of the virtual machines is executed, each of the virtual machines discontinues the operation thereof for a time period proportional to the size of the virtual machine. The virtual machine down time is the time period generated thereby.
The “migration stand-by time period” is a stand-by time period generated by the migration of the virtual machines. Each server receives the virtual machines one virtual machine at one time. The migration stand-by time period is a stand-by time period for the reception generated when many virtual machines are rearranged in one server.
In the seventh embodiment, the computer system 1 has the data N and S defining the configuration of the data center (
In the seventh embodiment, the computer system 1 may have the data defining the size of the virtual machines (
To add to the fifth embodiment, the computer system 1 of the seventh embodiment may include the data defining the service count (
Formulation of the mixed integer programming problem according to the seventh embodiment will be described. Contents identical to those of the modeling described in each of the first to the fifth embodiments will not again be described.
The penalty for the startup stand-by time period is expressed by Eq. (119).
The penalty for the virtual machine down time is expressed by Eq. (120).
In the above, “ym, n, i, j” is a binary variable satisfying Eq. (121) below.
y
m,n,i,j
:x
m,n,i,j
−x
m,n,i,j
0=1ym,n,i,j=1 (121)
x
m,n,i,j
−x
m,n,i,j
0≦0ym,n,i,j=0 (122)
“ym,n,i,j” takes a value of “one” when the n-th virtual machine in the m-th service moves to the j-th server in the i-th rack by the live migration, and takes a value of “zero” in other cases.
“Σym, n, i, j” in Eq. (120) takes a value of “one” when the n-th virtual machine in the m-th service is migrated by live migration, and takes a value of “zero” in other cases.
Therefore, Eq. (120) expresses the total penalty for the down time generated by the migration of the n-th virtual machine in the m-th service by the live migration. The down time is proportional to the size τmn.
“χ0m, n, i, j” is a constant that represents the initial arrangement of the virtual machine. Here, χ0m,n,i,j is χ0m,n,i,j=1 for n that is n≧Mm+1. “Mm” represents the number of virtual machines in the m-th service. In this case, the relation holds as below.
w
ij
0=Σm,nτmnxm,n,i,j0 (123)
The penalty for the migration stand-by time period is expressed by Eq. (124).
“uij” is an integer variable that satisfies Eqs. (125) and (126) below and that is greater than or equal to zero and less than or equal to Lij.
The left side of the inequality on the left of each of Eqs. (125) and (126) expresses the number of virtual machines migrated in the j-th server in the i-th rack. In Eq. (125), when the number of virtual machines migrated in the j-th server in the i-th rack is greater than or equal to one, uji takes the value of the left side of the inequality on the left side of Eq. (125). On the other hand, in Eq. (126), when the number of virtual machines migrated in the j-th server in the i-th rack is less than or equal to 1-1, uij takes a value of zero. Therefore, “eχuij” represents the penalty for the stand-by time period for the virtual machines of a number by which the threshold value l is exceeded of the virtual machines moving in the j-th server in the i-th rack. Therefore, Eq. (124) expresses the total thereof.
“ρij” is a binary variable that satisfies Eq. (127) below and is used to define “uij”.
Based on the above, the electric power minimization problem of the servers, the UPSs, the CRAC units, and the chiller plants by the rearrangement of the virtual machines can be formulated as a mixed integer programming problem as below.
Eq. (128) complies with the conditions expressed by Eqs. (129) to (135) in addition to the conditions expressed by Eqs. (88) to (107).
The mixed integer programming problem merely has to be solved such that the value of Eq. (128) is minimized. Eq. (128) is an equation formed by adding Eqs. (119), (120), and (124) to Eq. (87) described in the fifth embodiment. Eqs. (119), (120), and (124) may be applied to Eq. (36) of the second embodiment, Eq. (59) of the third embodiment, Eq. (71) of the fourth embodiment, and Eq. (117) of the sixth embodiment. Not all of the Eqs. (119), (120), and (124) need to be used and any one thereamong may be used.
As described, the mixed integer programming problem formulated in the seventh embodiment is solved by the solver such as, for example, GLPK, SYMPHONY, or Gurobi Optimizer. The control apparatus of the computer system executes the migration of the virtual machines according to the solution obtained by the solver and rearranges the virtual machines; and thereby, can facilitate the electric power saving of the data center with consideration of reductions in the startup stand-by time period, the virtual machine down time, and the migration stand-by time period, respectively.
According to the computer system and the virtual machine arranging method, an effect is achieved that electric power saving can be facilitated when plural virtual machines are assigned to plural servers.
All examples and conditional language provided herein are intended for pedagogical purposes of aiding the reader in understanding the invention and the concepts contributed by the inventor to further the art, and are not to be construed as limitations to such specifically recited examples and conditions, nor does the organization of such examples in the specification relate to a showing of the superiority and inferiority of the invention. Although one or more embodiments of the present invention have been described in detail, it should be understood that the various changes, substitutions, and alterations could be made hereto without departing from the spirit and scope of the invention.
Number | Date | Country | Kind |
---|---|---|---|
PCT/JP2011/071737 | Sep 2011 | JP | national |
PCT/JP2011/073973 | Oct 2011 | JP | national |
Number | Date | Country | |
---|---|---|---|
Parent | PCT/JP2012/073544 | Sep 2012 | US |
Child | 14223155 | US |