Patent Application
20130275601
The present invention relates generally to a computer implemented method, data processing system, and computer program product for integrating data processing systems having multiple hardware resources and multiple logical partitions (LPAR). More specifically, the present invention relates to a hypervisor mediating access to one or more hardware resources in a manner that the LPAR sends and receives signals relating to the hardware resource in a manner similar to communications direct to the hardware resource.
Virtualization is the creation of logical substitutes for physical resources. The substitutes have the same functions and external interfaces as their physical counterparts, but differ in attributes, such as size, performance, and cost. These substitutes are called virtual resources, and their users are typically unaware of the substitution. Virtualization is commonly applied to hardware resources by combining multiple hardware resources into shared pools from which a logical partition (LPAR) receives virtual resources. A hardware resource is a device that is physical. Examples of hardware resources include Ethernet adapters, I/O controller, audio adapters, and the like. In contrast, a virtual resource is the interface of the physical resource as presented to a logical partition. With virtualization, a computer system administrator can make one hardware resource perform as if it were multiple virtual resources.
A key software component supporting virtualization is the hypervisor. A hypervisor is used to logically partition the hardware into pools of virtualized resources known as logical partitions. Such logical partitions are made available to client entities, for example, operating systems and applications. Each logical partition of the hypervisor is unable to access hardware resources of a second logical partition unless such hardware resources are reassigned by the hypervisor. A hypervisor is sometimes known as a hosting partition.
Within a logical partition, an operating system may be stored. An OS partition is a logical partition in which an operating system is stored and executes. An operating system is used to perform basic tasks such as controlling and allocating memory, prioritizing system requests, controlling input and output devices, facilitating networking, and managing file systems. Such tasks are limited to the extent that the hypervisor allocates hardware resources to the operating system. Such hardware resources include input-output devices, and file storage adapters, and the like.
In addition to hardware resources enumerated above, a hypervisor may allocate I/O adapters. An I/O adapter is a physical network interface that provides memory-mapped input/output interface for placing queues into physical memory and provides an interface for control information. Control information can be, for example, a selected interrupt to generate when a data packet arrives. A data packet is a formatted block of data carried by a computer or communication network. A core function of the I/O adapter is handling the physical signaling characteristics of the network media and converting the signals arriving from the network to logical values. Depending on the type of I/O adapter, additional functional layers of the Open Systems Interconnection (OSI) model protocol stack may be handled within the I/O adapter, for example, the data link layer functions and the network layer functions, among others. In contrast, higher-level communication functions may be performed by the operating system to which the I/O adapter is assigned, or by applications within the operating system.
Servers are particularly dependent on the operation of I/O adapters to accomplish the functions of a server. In addition to providing data to users across a network, servers can draw attacks by malicious and unauthorized people.
The present invention provides a computer implemented method, data processing system, and apparatus for hardware resource arbitration in a data processing environment having a plurality of logical partitions. A hypervisor receives a request for a hardware resource from a first logical partition, wherein the request corresponds to an operation. The hypervisor determines the hardware resource is free from contention by a second logical partition. The hypervisor writes the hardware resource to a hardware resource pool data structure, as associated with the first logical partition, in response to a determination that the hardware resource is free. The hardware resource pool data structure may be controlled by the hypervisor. The hypervisor presents the hardware resource to the first logical partition. The hypervisor determines that the operation is complete. The hypervisor releases the hardware resource from a hardware resource pool, responsive to the determination that the operation is complete.
The novel features believed characteristic of the invention are set forth in the appended claims. The invention itself, however, as well as a preferred mode of use, further objectives and advantages thereof, will best be understood by reference to the following detailed description of an illustrative embodiment when read in conjunction with the accompanying drawings, wherein:
Data processing system 100 is a logical partitioned (LPAR) data processing system. Thus, data processing system 100 may have multiple heterogeneous operating systems or multiple instances of a single operating system running simultaneously. Each of these multiple operating systems may have any number of software programs executing within it. Data processing system 100 is logically partitioned such that different PCI I/O adapters 120-121, 128-129, and 136, graphics adapter 148, and hard disk adapter 149 may be assigned to different logical partitions. In this case, graphics adapter 148 connects a display device (not shown), while hard disk adapter 149 connects to and controls hard disk 150.
Thus, for example, suppose data processing system 100 is divided into logical partitions of virtual I/O server, P1, P2, and P3. Each of PCI I/O adapters 120-121, 128-129, 136, graphics adapter 148, hard disk adapter 149, each of processors 101-104, and memory from local memories 160-163 can be assigned to each of the partitions. In these examples, local memories 160-163 may take the form of dual in-line memory modules (DIMMs). DIMMs are not normally assigned on a per DIMM basis to partitions. Instead, a partition will be assigned a portion of the overall memory seen by the platform. For example, processors 102-103, some portion of memory from local memories 160-163, and PCI I/O adapters 121 and 136 may be assigned to logical partition P2; and processor 104, some portion of memory from local memories 160-163, graphics adapter 148 and hard disk adapter 149 may be assigned to logical partition P3.
Each operating system executing within data processing system 100 is assigned to a different logical partition. Thus, each operating system executing within data processing system 100 may access only those I/O units, which are assigned to its logical partition. Thus, for example, one instance of the Advanced Interactive Executive (AIX®) operating system may be executing within partition P1, a second instance or image of the AIX® operating system may be executing within partition P2, and a Linux® operating system may be operating within logical partition P3. AIX® is a registered trademark of International Business Machines Corporation. Linux® is a registered trademark of Linus Torvalds.
Peripheral component interconnect (PCI) host bridge 114 connected to I/O bus 112 provides an interface to PCI local bus 115. A number of PCI input/output adapters 120-121 connect to PCI bus 115 through PCI-to-PCI bridge 116, PCI bus 118, PCI bus 119, I/O slot 170, and I/O slot 171. PCI-to-PCI bridge 116 provides an interface to PCI bus 118 and PCI bus 119. PCI I/O adapters 120 and 121 are placed into I/O slots 170 and 171, respectively. Typical PCI bus implementations support between four and eight I/O adapters, that is, expansion slots for add-in connectors. Each PCI I/O adapter 120-121 provides an interface between data processing system 100 and input/output devices such as, for example, other network computers, which are clients to data processing system 100.
An additional PCI host bridge 122 provides an interface for an additional PCI bus 123. PCI bus 123 connects to a plurality of PCI I/O adapters 128-129. PCI I/O adapters 128-129 connect to PCI bus 123 through PCI-to-PCI bridge 124, PCI bus 126, PCI bus 127, I/O slot 172, and I/O slot 173. PCI-to-PCI bridge 124 provides an interface to PCI bus 126 and PCI bus 127. PCI I/O adapters 128 and 129 are placed into I/O slots 172 and 173, respectively. In this manner, additional I/O devices, such as, for example, modems or network adapters may be supported through each of PCI I/O adapters 128-129. Consequently, data processing system 100 allows connections to multiple network computers.
A memory mapped graphics adapter 148 is inserted into I/O slot 174 and connects to I/O bus 112 through PCI bus 144, PCI-to-PCI bridge 142, PCI bus 141, and PCI host bridge 140. Hard disk adapter 149 may be placed into I/O slot 175, which connects to PCI bus 145. In turn, this bus connects to PCI-to-PCI bridge 142, which connects to PCI host bridge 140 by PCI bus 141.
A PCI host bridge 130 provides an interface for a PCI bus 131 to connect to I/O bus 112. PCI I/O adapter 136 connects to I/O slot 176, which connects to PCI-to-PCI bridge 132 by PCI bus 133. PCI-to-PCI bridge 132 connects to PCI bus 131. This PCI bus also connects PCI host bridge 130 to the service processor mailbox interface and ISA bus access pass-through logic 194 and PCI-to-PCI bridge 132. Service processor mailbox interface and ISA bus access pass-through logic 194 forwards PCI accesses destined to the PCI/ISA bridge 193. NVRAM storage 192, also known as non-volatile RAM, connects to the ISA bus 196. Service processor 135 connects to service processor mailbox interface and ISA bus access pass-through logic 194 through its local PCI bus 195. Service processor 135 also connects to processors 101-104 via a plurality of JTAG/I2C busses 134. JTAG/I2C busses 134 are a combination of JTAG/scan busses, as defined by Institute for Electrical and Electronics Engineers standard 1149.1, and Philips I2C busses. However, alternatively, JTAG/I2C busses 134 may be replaced by only Philips I2C busses or only JTAG/scan busses. All SP-ATTN signals of the processors 101, 102, 103, and 104 connect together to an interrupt input signal of service processor 135. Service processor 135 has its own local memory 191 and has access to the hardware OP-panel 190.
When data processing system 100 is initially powered up, service processor 135 uses the JTAG/12C busses 134 to interrogate the system processors 101-104, memory controller/cache 108, and I/O bridge 110. At the completion of this step, service processor 135 has an inventory and topology understanding of data processing system 100. Service processor 135 also executes Built-In-Self-Tests (BISTs), Basic Assurance Tests (BATs), and memory tests on all elements found by interrogating processors 101-104, memory controller/cache 108, and I/O bridge 110. Any error information for failures detected during the BISTs, BATs, and memory tests are gathered and reported by service processor 135.
If a meaningful or valid configuration of system resources is still possible after taking out the elements found to be faulty during the BISTs, BATs, and memory tests, then data processing system 100 is allowed to proceed to load executable code into local memories 160-163. Service processor 135 then releases processors 101-104 for execution of the code loaded into local memory 160-163. While processors 101-104 are executing code from respective operating systems within data processing system 100, service processor 135 enters a mode of monitoring and reporting errors. The type of items monitored by service processor 135 includes, for example, the cooling fan speed and operation, thermal sensors, power supply regulators, and recoverable and non-recoverable errors reported by processors 101-104, local memories 160-163, and I/O bridge 110.
Service processor 135 saves and reports error information related to all the monitored items in data processing system 100. Service processor 135 also takes action based on the type of errors and defined thresholds. For example, service processor 135 may take note of excessive recoverable errors on a processor's cache memory and determine that this condition is predictive of a hard failure. Based on this determination, service processor 135 may mark that processor or other system resource for de-configuration during the current running session and future Initial Program Loads (IPLs). IPLs are also sometimes referred to as a “boot” or “bootstrap.”
Data processing system 100 may be implemented using various commercially available computer systems. For example, data processing system 100 may be implemented using IBM eServer iSeries Model 840 system available from International Business Machines Corporation. Such a system may support logical partitioning, wherein an OS/400® operating system may exist within a partition. OS/400 is a registered trademark of International Business Machines Corporation.
Those of ordinary skill in the art will appreciate that the hardware depicted in
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.
Any combination of one or more computer readable storage device(s) may be utilized. A computer readable storage device may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage device would include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage device may be any tangible device that can store a program for use by or in connection with an instruction execution system, apparatus, or device. The term “computer-readable storage device” does not encompass a signal propagation media such as a copper cable, optical fiber or wireless transmission media.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
Aspects of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The aspects of the illustrative embodiments provide a computer implemented method, data processing system, and computer program product for hardware resource arbitration in a data processing system having a several logical partitions. The embodiments permit a device driver to be used in place of a virtual device driver where a hypervisor maintains ownership of each hardware resource. In addition, a virtual I/O server may virtualize each hardware in a hardware resource pool.
Although the hypervisor may assign a hardware resource to a logical partition (LPAR), the hypervisor controls such assignment, and accordingly owns the hardware resource. Illustrative embodiments may show a re-allocation assignment of a hardware resource between contending partitions. Hypervisor 205 can provide I/O to the LPARs in a manner to emulate the physical signals of each virtual resource. Accordingly, each logical partition, P1 211, P2 213 and P3 215, relies on their respective device drivers, device driver 211, device driver 213, and device driver 215. A device driver is a processor executing computer instructions to interface a higher-level computer program to a hardware device. The device driver adapts non-uniform hardware architectures into a set of features that, by convention, permit interchangeable higher-level computer programs to access functionality of the hardware. As such, the device driver acts as an abstraction layer between a hardware device and an LPAR. Accordingly, hardware manufacturers, having an appropriate device driver performing as an intermediary, are able to provide devices that can work with a wide range of partitions organized by varying operating systems.
In contrast, a virtual device driver is a program that responds to software interrupts from an LPAR partition. The software interrupts correspond to requests and responses to data traversing a physical resource. The software interrupt occurs in response to an instruction executed on a processor, which can cause a context switch to occur. A further distinction between a virtual device driver and a device driver is that a virtual device driver is written specifically to interact with a virtual I/O path in the Hypervisor, while a physical device driver interacts directly with the hardware itself.
In order to coordinate the allocation of a hardware resource to an LPAR, the hypervisor stores current hardware resource assignments to hardware resource pool data structure 209. A hardware resource pool data structure establishes assignments of a hardware resource to a partition, and may also assign a priority to the assignment. The hardware resource pool data structure is controlled by the hypervisor. Moreover, the hardware resource pool data structure may have multiple LPARs assigned to a hardware resource. If multiple LPARs are assigned a hardware resource, the hypervisor may alternate connections between a first LPAR and a second LPAR according to a priority assignment value. A priority assignment value is a proportionate amount of time to which a device is entitled full, though indirect, control of a hardware resource. Such a proportion may be in relation to a period of time during which a hardware resource may be shared among LPARs. A proportion for an LPAR may be assigned by taking into account the number of LPARs contending for the device, as well as the relative priority assigned to each LPAR.
In the instance where the hypervisor responds that the hardware resource is busy, the error may be one that is typical in an environment where a partition has exclusive access to the hardware resource. An error is a signal or message that reports that a secondary state has occurred with respect to a target device. The secondary state is a sub-optimal state that can indicate a lower level of functionality than an optimal state. For example, an error when requesting read access to a hard drive may be a signal that indicates that the hard drive, or target device, is busy. Another error may be that which occurs when the transfer queue is full-namely dropping a packet. In contrast, an optimal state for the hard drive may include a signal that includes an acknowledgment or a buffer or other stream containing data retrieved from the hard drive. Such a signal may be transmitted within hardware resource R1 allocated 405.
In each case above, signals sent to the logical partition 450 are processed by the device driver that operates within the logical partition. Similarly, the signals sent to hypervisor 400 are sent from the device driver. In other words, the code of the device driver executing on the processor sends and receives signals in response to asynchronous events outside the LPAR, and responsive to software interrupts from within the LPAR. Accordingly, each such resource communication, as well as the communications of the data accessed thereby, are handled by a device driver that is not a virtual device driver.
Next, the hypervisor may determine whether the hardware resource is free (step 505). If the hardware resource is not free, a negative result occurs. Accordingly, the hypervisor may generate an error to LPAR (step 521). Processing may terminate thereafter.
Alternatively, the hypervisor may reach a positive result to step 505. Accordingly, the hypervisor may write the hardware resource as being associated with the LPAR (step 507). This step can include writing a row to a hardware resource pool data structure, such as hardware resource pool data structure 300 of
Responsive to the operation completing, the hypervisor may release the resource by writing the hardware resource as free to the resource pool (step 515). Processing may terminate thereafter.
If the hardware resource is free, the hypervisor may write the hardware resource as associated with LPAR (step 607). Next, the hypervisor may present the hardware resource to the LPAR (step 609). Attendant with presenting the hardware resource to the LPAR, the hypervisor may initiate a timer to detect if a priority assignment value has expired with relation to the hardware resource/LPAR assignment, providing the hardware resource is contended for by a second LPAR. The hypervisor may continue by determining if the operation is complete (step 613). Responsive to the operation completing, the hypervisor may release the hardware resource by writing the hardware resource as free to the hardware resource pool (step 615). Processing may terminate thereafter.
A negative result at step 613 can cause the hypervisor to determine if the requested hardware resource is contended for and whether a priority assignment value has expired (step 631). The priority assignment value, explained above, corresponds to a priority assignment value corresponding to the hardware resource/LPAR assignment. Step 631 is performed with reference to a priority assignment value, e.g., “1” 331 of
At steps 509, 515, 521 (of
The illustrative embodiments permit software developers and/or system integrators working with a virtualized environment to assemble program loads of operating systems and device drivers to pair hardware resources to the LPARs without resorting to authoring, installing, initializing and otherwise maintaining a specialty virtual device driver for each device used as a hardware resource in the data processing system.
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The invention can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements. In a preferred embodiment, the invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.
Furthermore, the invention can take the form of a computer program product accessible from a computer-usable or computer-readable storage device providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable storage device can be any tangible apparatus that can store the program for use by or in connection with the instruction execution system, apparatus, or device.
The computer-readable storage device can be an electronic, magnetic, optical, electromagnetic, or semiconductor system (or apparatus or device). Examples of a computer-readable storage device include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read/write (CD-R/W) and DVD.
A data processing system suitable for storing and/or executing program code will include at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories, which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.
Input/output or I/O devices (including but not limited to keyboards, displays, pointing devices, etc.) can be coupled to the system either directly or through intervening I/O controllers.
Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters.
The description of the present invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiment was chosen and described in order to best explain the principles of the invention, the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12570127 | Sep 2009 | US |
| Child | 13914101 | US |
