Patent Application
20120066676
This disclosure relates to disabling circuitry from initiating modification, at least in part, of information associated, at least in part, with at least one state of the circuitry.
In one conventional virtualization arrangement, a computer executes virtualization software and operating systems. The computer includes physical hardware resources that are shared by the virtualization software among the operating systems. The resulting shared resources may be associated with the operating systems as virtual machines. As can be appreciated, the virtualization software imposes processing and latency overhead. This reduces performance. In an effort to reduce such overhead, certain virtualization software permits a pass through mechanism whereby at least some of the physical hardware resources may be directly assigned to and accessed by one or more operating systems in a manner that bypasses the virtualization software.
One perceived benefit of virtualization is the relative ease with which a given virtual machine may be relocated from one physical computer to another physical computer, with little to no service down time. This largely results from the ability of the virtualization software to copy and restore respective virtual machine states between given virtual machines.
However, this presupposes that the virtualization software maintains current information concerning and is able to control the internal states of the physical hardware resources. Therefore, the use of pass through mechanisms complicates this process. This is because, by design, interactions between the pass through hardware resource and the operating system assigned to the resource, and control of the resource itself, bypass the virtualization software. Therefore, the internal states of the hardware resource may be unknown or unavailable to the virtualization software, and/or the internal states may be changing and/or subject to change during the migration process.
It has been proposed to use teamed hot plug pass through devices as a solution to the above problems. In this proposed solution, a physical pass through resource is bound to a software virtualized device using a teaming driver mechanism in the operating system. The pass through resource may be hot plugged off from the operating system, before migration, via a virtual or physical hot plug event. Unfortunately, in addition to involving use of operating system hot plug features, this proposed solution also suffers to the disadvantage of being relatively slow to execute.
Another proposed solution also involves use of another type of operating system driver, and suffers from similar disadvantages. Yet another proposed solution involves placing the virtualization software into an emulation mode that is tailored to the specific physical resource involved in the migration. Given the many different types, functions, features, and operations of physical resources that could potentially be involved in migration operations, this greatly complicates design of the virtualization software.
Features and advantages of embodiments will become apparent as the following Detailed Description proceeds, and upon reference to the Drawings, wherein like numerals depict like parts, and in which:
Although the following Detailed Description will proceed with reference being made to illustrative embodiments, many alternatives, modifications, and variations thereof will be apparent to those skilled in the art. Accordingly, it is intended that the claimed subject matter be viewed broadly.
Host computer 10 may comprise one or more single and/or multi-core host processors (HP) 12 and computer-readable/writable memory 21. Although not shown in the Figures, host computer 10 also may comprise one or more chipsets (comprising, e.g., memory and/or input/output controller circuitry). One or more host processors 12 may be communicatively coupled via the one or more chipsets to memory 21 and one or more resources 504. One or more resources 504 may comprise one or more pass through devices 72.
One or more pass through devices 72 may comprise network interface and/or input/output (I/O) controller circuitry (NIC) 150. In an embodiment, network and/or I/O controller circuitry may be or comprise circuitry capable of monitoring, facilitating, controlling, permitting, and/or implementing, at least in part, transmission, storage, retrieval, and/or reception, at least in part, of one or more symbols and/or values. Circuitry 150 may be communicatively coupled to host computer 20 via one or more networks 50. Additionally or alternatively, NIC 150 may be or comprise storage, mass storage, and/or disk storage controller circuitry. In this embodiment, circuitry 150 may comprise circuitry 118. Circuitry 118 may comprise, for example, computer readable/writable memory 25.
Alternatively or additionally, although not shown in the Figures, some or all of circuitry 150, 118, and/or the functionality and components thereof may be comprised in, for example, one or more host processors 12 and/or the one or more not shown chipsets. Also alternatively, one or more host processors 12, memory 21, the one or more not shown chipsets, and/or some or all of the functionality and/or components thereof may be comprised in, for example, circuitry 150 and/or circuitry 118. Many other alternatives are possible without departing from this embodiment.
Depending upon, for example, the particular embodiment, host 20 may comprise, in whole or in part, respective components and/or functionality that may be similar to the respective components and/or functionality of host 10. However, alternatively or additionally, host 20 may comprise, at least in part, respective components and/or functionality that may differ, at least in part, from respective components and/or functionality of host 10. For example, host computer 20 may comprise one or more host processors 12′, memory 21′, one or more pass through devices 74, and/or circuitry 118′ that may be generally similar in respective function and/or respective operation, at least in part, to one or more host processors 12, memory 21, one or more devices 72, and/or circuitry 118.
As used herein, “circuitry” may comprise, for example, singly or in any combination, analog circuitry, digital circuitry, hardwired circuitry, programmable circuitry, co-processor circuitry, state machine circuitry, and/or memory that may comprise program instructions that may be executed by programmable circuitry. Also in this embodiment, a processor, processor core, core, and controller each may comprise respective circuitry capable of performing, at least in part, one or more arithmetic and/or logical operations, such as, for example, one or more respective central processing units. Also in this embodiment, a chipset may comprise circuitry capable of communicatively coupling, at least in part, one or more host processors, storage, mass storage, one or more nodes, and/or memory. Although not shown in the Figures, host computer 10 and/or host computer 20 each may comprise a respective graphical user interface system. The not shown respective graphical user interface systems may comprise, e.g., respective keyboards, pointing devices, and display systems that may permit one or more human users to input commands to, and monitor the operation of, host computer 10, node 20, and/or system 100.
One or more machine-readable program instructions may be stored in computer-readable/writable memory 21 and/or 25. In operation of host computer 10, these instructions may be accessed and executed by one or more host processors 12, one or more devices 72, NIC 150, and/or circuitry 118. When executed by one or more host processors 12, one or more devices 72, NIC 150, and/or circuitry 118, these one or more instructions may result in one or more operating system environments (OSE) 70A . . . 70N, and/or one or more virtual machine monitors (VMM) 35 being executed at least in part by one or more host processors 12 and becoming resident at least in part in memory 21. Also when executed by one or more host processors 12, one or more devices 72, NIC 150, and/or circuitry 118, these one or more instructions may result in one or more host processors 12, one or more devices 72, NIC 150, circuitry 118, one or more OSE 70A . . . 70N, and/or one or more VMM 35, performing the operations described herein as being performed by these components of system 100. In this embodiment, a portion of an entity may comprise all or less than all of the entity. In this embodiment, an operating system environment or guest operating system environment may be used interchangeably and may comprise one or more portions of one or more operating systems and/or one or more applications. Also in this embodiment, a virtual machine monitor may comprise, at least in part, one or more processes capable of monitoring and/or controlling, at least in part, operation of one or more operating system environments, such as, for example, one or more hypervisor processes. Also, in this embodiment, a process, program, driver, operating system, and application may be used interchangeably, and may comprise and/or result at least in part from execution of one or more program instructions and/or software threads. Although not shown in the Figures, one or more processes 42 may be comprised, at least in part, in one or more virtual machine monitors 42, or vice versa. Also in this embodiment, memory 21 and/or memory 25 may comprise one or more of the following types of memories: semiconductor firmware memory, programmable memory, non-volatile memory, read only memory, electrically programmable memory, random access memory (such as, for example, dynamic and/or static random access memory), flash memory, magnetic disk memory, optical disk memory, register memory, cache memory, memory to store state information, and/or other or later-developed computer-readable and/or writable memory. Also, in this embodiment, a resource may comprise, at least in part, circuitry and/or one or more processes, such as, without limitation, one or more I/O resources and/or memory. Additionally, in this embodiment, a pass through device may comprise, at least in part, a resource that is capable of being accessed and/or controlled, at least in part, by at least one portion of an operating system environment in such a way that involvement of a virtual machine monitor in such access and/or control is bypassed, at least in part.
In this embodiment, a state or operational state of an entity may be used interchangeably, and may be or comprise, at least in part, (1) internal or external contents, context, condition, operation, function, attribute, instrumentality, calculation and/or activity of, being executed by, executable by, and/or associated with, at least in part, the entity, and/or (2) data and/or information describing, related to, involved in, comprised in, associated with, useful for, facilitating, and/or based upon, directly or indirectly, at least in part, such contents, context, condition, operation, function, attribute, instrumentality, calculation, and/or activity. Particular examples of states and/or state information in this embodiment may include software, hardware, process, communication, protocol, and input/output context information, value of in-flight and/or intermediate calculations, iterative convergence and/or equation variable-related data, open files and/or connections, etc. Also, in this embodiment, a mode of operation of an entity may result from, be associated with, comprise, and/or involve, at least in part, one or more states of the entity.
One or more machine-readable program instructions may be stored in computer-readable/writable memory 21′. In operation of host computer 20, these instructions may be accessed and executed by one or more host processors 12′, one or more devices 74, one or more resources 526, and/or circuitry 118′. When executed by one or more host processors 12′, one or more devices 74, one or more resources 526, and/or circuitry 118′, these one or more instructions may result in one or more OSE 75A . . . 75N, and/or one or more VMM 512 being executed at least in part by one or more host processors 12′ and becoming resident at least in part in memory 21′. Also when executed by one or more host processors 12′, one or more devices 74, one or more resources 526, and/or circuitry 118′, these one or more instructions may result in one or more host processors 12′, one or more devices 74, one or more resources 526, circuitry 118′, one or more OSE 75A . . . 75N, and/or one or more VMM 512, performing the operations described herein as being performed by these components of system 100. In this embodiment, one or more VMM 526 may comprise one or more processes 520.
With reference now being made to
Also, in operation of system 100, the execution of one or more VMM 512 may result, at least in part, in host 20 comprising, at least in part, one or more (and in this embodiment, a plurality of) VM 510A . . . 510N. For examples, one or more VM 510A may comprise one or more OSE 75A, and one or more resources 526A that may be accessible and/or controllable, at least in part, by one or more OSE 75A. One or more VM 510N may comprise one or more OSE 75N, and one or more resources 526N that may be accessible and/or controllable, at least in part, by one or more OSE 75N. In this embodiment, one or more resources 526A may comprise, at least in part, circuitry 118′.
As shown in
As a result, at least in part, of one or more commands issued to host computer 10, host computer 20, and/or system 100 (e.g., via the one or more not shown user interfaces), one or more processes 42 may issue, at least in part, to circuitry 118 one or more commands (CMD) 30 (see
Prior to receiving, at least in part, one or more commands 30, circuitry 118 may operate in a first mode of operation 302 in which circuitry 118 may be enabled to modify, at least in part, information 22 that may be maintained, at least in part, by circuitry 118, as illustrated by operation 402 in
In response, at least in part, to one or more commands 30, circuitry 118 may undergo a transition 301 to and enter a second mode of operation 304 (see
For example, as shown in
Also, for example, in migrating, at least in part, one or more VM 502A from host 10 to host 20, one or more processes 520 in VMM 512 may utilize, at least in part, the information transferred, at least in part, by one or more processes 42 to establish, make resident, and/or permit to be executed, at least in part, in host 20, one or more VM 502A. For example, one or more processes 520 may store information 22 in one or more not shown memory regions in host 20. Circuitry 118′ in host 20 may replace, at least in part, current state information (not shown) in circuitry 118′ with information 22. As a result, at least in part, of circuitry 118′ so replacing, at least in part, this current state information with information 22, circuitry 118′ and/or one or more resources 526A may enter the one or more operating states that circuitry 118 was operating in immediately prior to receiving one or more commands 30. These operations may result, at least in part, in VM 502A becoming resident and/or being executed in host 20, and in circuitry 118′ and/or one or more resources 526A becoming, at least in part, associated with VM 502A as one or more pass through devices of guest OSE 70A in host 20. Thus, as a result, at least in part, of these operations, circuitry 118′ and/or one or more resources 526A may be re-assigned, at least in part, by VMM 512 to operate, at least in part, as one or more pass through devices associated, at least in part, with guest OSE 70A in VM 502A in host 20.
In this embodiment, circuitry 118 may perform operations 306 and/or 308, at least in part, via one or more direct memory access operations. For example, information 22 may be transferred, at least in part, by circuitry 118 to one or more regions 40 via direct memory access. Also, for example, information 52 may be transferred, at least in part, by circuitry 118 from one or more regions 40 via one direct memory access. Alternatively or additionally, the transfers of information 22 and/or 52 may be accomplished at least in part via, for example, one or more memory reads carried out by one or more HP 12 and/or one or more processes 42. Further alternatively or additionally, such transfers may be accomplished, at least in part, by a combination of memory read/write operations. For example, one or more processes 42 may export to NIC 150 one or more internal addresses and/or data sizes to be read, and NIC 150 may return to one or more processes 42 the data satisfying these addresses and/or data sizes in a subsequent read thereof by one or more processes 42. Of course, other memory transfer techniques may be utilized without departing from this embodiment. Accordingly, operation 306 and/or operation 308 should be viewed as encompassing all such possible memory transfer techniques.
Also in this embodiment, in response, at least in part, to one or more commands 30, circuitry 118 may execute transition 301, operation 306, operation 308, and/or transition 303 in a single atomic operation. Thus, for example, this single atomic operation may comprise entry by the circuitry 118 into the second mode 304 from the first mode 302, copying, at least in part, of information 22 to the one or more regions 40, and/or replacing, at least in part, information 22 in memory 25 with information 52. In this embodiment, an atomic operation may comprise a plurality of operations intended to be executed completely, and as a single transaction.
In this embodiment, when circuitry 118 is operating in the second mode 304, one or more internal states (e.g., one or more states related to the migration) of circuitry 118 may no longer be capable of being changed, except via operation 308. Additionally, in this embodiment, transition 301 and/or transition 303 may be achieved, at least in part, via one or more relatively fast internal state changes (e.g., involving circuitry 118). In this embodiment, when circuitry 118 transitions to mode 304, one or more not shown in-process packets and/or contents of one or more not shown internal packet first-in-first-out buffers may be dropped and/or flushed. Also, in this embodiment, when circuitry 118 transitions to mode 304, one or more on-going direct memory access operations may be terminated or suspended until circuitry 118 returns to mode 302.
Additionally, with reference to
Also, although not shown in the Figures, NIC 150 may be or comprise a circuit card to be coupled to a circuit board that may include, for example, HP 12, memory 21, and/or the one or more not shown chipsets in host 10. Additionally or alternatively, NIC 150 may be comprised, at least in part, in one or more not shown integrated circuit chips. Further additionally or alternatively, NIC 150 may comprise, at least in part, one or more virtual functions that comply and/or are compatible, at least in part, with “Single Root I/O Virtualization and Sharing Specification,” Revision 1.0, Sep. 11, 2007, PCI-SIG, Beaverton, Oreg., USA.
Thus, an embodiment may include circuitry to be comprised at least in part in a first host, and being enabled, when the circuitry is in a first mode of operation, to modify, at least in part, first information maintained, at least in part, by the circuitry and associated, at least in part, with at least one operational state. The circuitry may be disabled from initiating modification to the first information when the circuitry is in a second mode. The circuitry may enter the second mode in response to at least one command. When in the second mode, the circuitry may (1) copy, at least in part, the first information to at least one memory region, (2) replace, at least in part, the first information with second information, and (3) enter at least another operational state associated, at least in part, with the second information.
Advantageously, in this embodiment, relocation and/or migration of a virtual machine from one physical host to another physical host may be carried out more easily, with little or no service down time, despite use by the virtual machine of one or more pass through devices. This may result, at least in part, from, among other things, the disabling of the circuitry 118, when in the second mode of operation 304, from being able to initiate modification to information 22. This may freeze modification to such information 22 in this embodiment until after circuitry 118 carries out an atomic operation that copies information 22 to one or more regions 40 (e.g., as a result of operation 306), replaces, at least in part, information 22 with information 52 in memory 25, and re-enters the first mode of operation 302. Thus, in this embodiment, information 22 may be available to one or more processes 42 of VMM 35 to implement the relocation and/or migration. Additionally, the VM migration and/or relocation in this embodiment is relatively fast to execute, and does not necessarily involve a virtual or physical hot plug event, the guest operating system environment to carry out the migration and/or relocation, and placing the virtualization software into an emulation mode.
Many modifications, alternatives, and variations are possible without departing from this embodiment. Accordingly, the claims should be viewed broadly as encompassing all such modifications, alternatives, and variations.
