Patent Grant
8799903
This application is related to U.S. patent application Ser. No. 11/888,343, filed on Jul. 31, 2007, and entitled, “MANAGEABILITY PLATFORM IN AN UNIFIED SYSTEM, now U.S. Pat. No. 8,185,913 and U.S. patent application Ser. No. 11/888,348, filed on Jul. 31, 2007, and entitled, “STORAGE-CENTRIC MANAGEABILITY IN A SYSTEM”, now U.S. Pat. No. 7,757,023, which are herein incorporated by reference in their entireties.
Manageability is a key requirement for a broad spectrum of information technology (IT) systems ranging from laptops to blade servers to clusters to large scale data centers. With rising complexity and scale in tomorrow's enterprise IT, system manageability has become a dominating cost. As referred herein, manageability includes management and maintenance tasks or operations that deal with bringing up, maintaining, tuning, and retiring a system. Also referred herein, and as understood in the art, information technology, or IT, encompasses all forms of technology, including but not limited to the design, development, installation, and implementation of hardware and software information or computing systems and software tasks, used to create, store, exchange and utilize information in its various forms including but not limited to business data, conversations, still images, motion pictures and multimedia presentations technology and with the design, development, installation, and implementation of information systems and tasks. Thus, examples of IT management and maintenance tasks or operations include diagnostics and recovery, security protection, backups, resource provisioning, and asset management of IT systems.
At a broader level, the scope of IT manageability may be associated with the lifecycle phases for servers and data centers, including bring up, operation, failures/changes, and retire/shutdown phases. Various manageability tasks are performed at each of these life cycle stages. Examples include provisioning and installation of servers, monitoring performance and health of systems, security protection against viruses and spyware, backup protection against disasters, disk maintenance to improve performance, fault diagnostics and recovery, and asset management to track resources. Several efforts are underway to address this growing problem of manageability. For example, software based solutions have been proposed to address manageability at the different lifecycle phases. In such solutions, several of the manageability tasks execute during the operation phase of the servers, sharing hardware and software resources with host applications. This sharing leads to resource interference and hence degradation in performance. Such degradation is expected to worsen with growing IT complexity and corresponding increases in the growing number and sophistication of manageability tasks.
One approach to addressing the above concerns is to provide better platform support for manageability tasks. An emerging trend towards this direction is the use of manageability processors (MPs)—dedicated hardware processors that only execute manageability tasks and provide an out-of-band channel for remote management. A typical MP is a small embedded application-specific integrated circuit (ASIC) customized for specific manageability uses or operations. It can be hooked off, for example, the peripheral component interconnect (PCI) bus at an input/output (I/O) bus (e.g., the southbridge) of computerized systems such as servers and personal computers (PCs). Instantiations of such MP architectures or platforms follow an asymmetrical model. The host system includes a powerful processor or central processing unit (CPU), large memory, network interface cards or modules (NIC), a server operating system (OS), while the manageability system typically includes a cheaper embedded processor, a small dedicated memory, NIC, and a private embedded OS that executes independently of the host system. Such asymmetry and independence aids in removing resource interference for processors, buses, caches, and memory, thereby resulting in improved performance for host workloads that are CPU and memory bound.
Thus, it would be beneficial to provide a platform support for manageability tasks that seamlessly interact with a host system for which the manageability tasks are performed.
Embodiments are illustrated by way of example and not limited in the following figure(s), in which like numerals indicate like elements, in which:
For simplicity and illustrative purposes, the principles of the embodiments are described by referring mainly to examples thereof. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments. It will be apparent however, to one of ordinary skill in the art, that the embodiments may be practiced without limitation to these specific details. In other instances, well known methods and structures have not been described in detail so as not to unnecessarily obscure the embodiments.
U.S. patent application Ser. No. 11/888,343, now U.S. Pat. No. 8,185,913 and U.S. patent application Ser. No. 11/888,348, now U.S. Pat. No. 7,757,023 provide various embodiments for using a dedicated computing element, such as a manageability processor, that may be a part of a dedicated manageability platform for handling manageability tasks that are delegated away from a host system. Such delegation of system manageability provides several benefits.
First, such delegation improves host application performance. Delegating the manageability applications to a separate processing element eliminates resource contention and interference at all higher levels of the system, including at the thread level, at the shared caches' level, and at the memory bus level. Furthermore, having a separate OS stack for such a processing element also eliminates software level contention. This allows for a more powerful policy engine, such as the policy system 809, that enables more intelligent arbitration between the host applications and manageability applications to improve performance.
Second, delegating the manageability applications improves host power efficiency in certain cases. For example, compared to the host processor, the delegated separate computing element may be smaller, and consequently, the system architecture embodiments as described herein are likely to be more power efficient compared to traditional approaches. Isolating the two applications also provide greater opportunity for dynamic power control techniques like voltage and frequency scaling.
Third, delegating the manageability applications away from the host system enable separation of the manageability and host application domains for control and security. As discussed earlier, administrators often prefer additional control on manageability applications to have stricter controls on disabling or changing parameters. (For example, disabling virus scanning by a system user to improve performance might be disastrous.) From a fault-tolerance perspective, again, having separate hardware fault domains for the manageability and host processing provides valuable benefits, such as isolating failures due to software errors, hardware errors, security attacks so that these failure do not spread from one domain to another.
Fourth, delegating the manageability applications away from the host system enables use of an out-of-band power domain for manageability in certain cases so that system manageability may be performed while the host system is set to a power standby or off state to conserve power.
Fifth, delegating the manageability applications away from the host system enables portability of the manageability functionality. For example, it may be desirable to have a common manageability solution across different kinds of systems.
Sixth, from the perspective of a manageability application, delegating the manageability applications to a delegated computing element away from the host system also provide several benefits. For example, given its own local processing, many manageability tasks may be run 24×7 in the background all the time without having to deal with issues around resource contention. For applications like security or data integrity, this can be an important issue. Also, the proximity of the delegated computing element to the storage subsystem may potentially reduce the I/O latency, further improving the manageability performance.
Accordingly, the decoupled software architecture, as provided by having two separate operating systems to run on the host system and the manageability platform, eliminates contention for resources such as those of processors, hardware caches, and memory bus. Eliminating such hardware contention does provide improved performance for CPU and memory bound host applications when executed in parallel with manageability applications.
Several manageability tasks or workloads rely on the data from the host system for its operations and may generate I/O contention at the shared disk with the host application workload. Examples include backup, anti-virus scan, disk auditing, disk indexing. This I/O contention, in turn, may affect the I/O path performance. Two major factors in the I/O path performance are disk seek times and the buffer cache hit rates. In a single OS image environment, the OS maintains an I/O queue where requests from multiple processes are sorted and maintained in an order such that disk seek times are reduced. A large body of work has gone into the design of such algorithms and software stacks which have helped improve performance of applications drastically. However, in manageability system frameworks with multiple independent and private OS images, multiple such I/O queues are being maintained for the shared data storage area in the host system, such as a shared disk drive. Each of these OS'es schedule the I/O blocks from their own private queue with no information about the I/O block access patterns at the other processor. Because the disk is still shared, this results in the I/O controller and disk to effectively receive an unsorted and at times random order of I/O requests that result in very poor disk seek performance due to back and forth movement of the disk head.
The OS also maintains a buffer cache that caches disk contents for better performance. Existing manageability architectures maintain separate buffer caches in their own private OS images. Maintaining separate private buffer caches at the host and manageability systems helps prevent unwanted cache eviction for host workloads. However, where the disk blocks overlap, the two systems would now miss the advantage of mutual cache hits. Finally, maintaining separate file system stacks in multiple OS images lead to consistency issues. As mentioned above, the OS maintains a buffer cache and if pages are dirty in multiple buffer caches, there is a huge file consistency problem in such decoupled software architectures.
Accordingly, it is desirable to eliminate the above possible contention for shared input/output (I/O) to a data storage area, such as shared disk drive.
Described herein are systems and methods for coupling OS pairs in a system framework or architecture that exhibits a decoupled software architecture therein, such as a manageability system framework with multiple independent and private OS'es, so as to minimize or eliminate I/O contention for shared data storage space, such as a shared disk drive.
From a hardware perspective, the processing system such as the manageability system 110 includes a processing element 112 with supporting read only memory (ROM) 114 and RAM 116. In one embodiment, the processing element 112 is a general-purpose processor of any of a number of computer processors, such as processors from Intel, AMD, and Cyrix, that is operable to execute a myriad of manageability tasks instead of the typical task specific integrated circuits (ASICs) that are currently used in some computer systems for limited manageability functions. Thus, in one example, the processing element 112 is a manageability processor (MP) dedicated to executing manageability tasks in the host system, freeing up the main or host CPU 101 to execute host system tasks with higher performance. However, alternative examples are contemplated wherein the MP is a processor in a multi-processor host system, a core in a multi-core processor implemented in the host system, an enhanced computer processor implemented in a disk controller for a data storage area in the host system.
As illustrated in
From a software perspective, the software model for the manageability system 110 includes two main components that may installed as firmware or software in the ROM 414 and resident in RAM 116 at runtime. The first component is a general software system that includes an embedded OS acting as core controller to host manageability software tasks and a set of standard software interfaces, such as those defined by Platform Management Components Interconnect (PMCI) Working Group of the Distributed Management Task Force (DMTF). The second component of the software model comprises the middleware that pertains to synchronization of information across the host and manageability environments and other application specific interfaces (APIs) between host applications 201 and manageability applications 205.
At the lowest layer, the block I/O layers 204 and 208 of the two systems are loosely coordinated. The I/O queues are exchanged between them, and the disk block requests (in the I/O queue) at the manageability system 230 are reordered periodically to follow that of the host system 220 for issuance to the I/O controller 209 to access the shared disk 210. Thus, the disk seek times are expected to improve. Next, at the file system levels or layers 203 and 207, the read/write consistency is coordinated for file system coupling. This is done by pre-selecting the host system 220 to be the sole master responsible for all writes in the overall system architecture 200. The manageability system 230 only performs reads and redirects all writes to the host system 220. In addition, the host system 220 is responsible for electing the type of file system to be used for the shared data. This is communicated to the manageability system 230 which follows the host system 220. It should be noted that the I/O block layer and the file system layer are parts of an OS.
In addition to the block I/O coordination or coupling, there is provided synchronization of file access patterns from the host system 220 to the management system 230 for file system access coupling. This is depicted at the system middleware layers 202 and 206, as illustrated in
The block I/O, file system, and file system access couplings in an execution flow of the overall system architecture 200 are provided through daemons at the host system 220 and the manageability system 230. During system startup, a daemon at the host system 220 provides the information regarding the file system type of the shared disk (108-110,
There are two key components of block I/O coupling. One is at the host system 220 with regards to creating a dump of the I/O queue. The other is at the manageability system 230 with regard to reordering its block I/O queue given periodic dumps from the host system 220.
The method 300 implements a windowing mechanism to obtain a snapshot of the I/O queue as seen appropriate for coordination by the I/O scheduler (not shown) in the host OS. This is done periodically, for example, every t seconds, with the waking up or activation of the host daemon (at 310). The I/O queue may be looked upon as containing a continuous stream of I/O requests that are issued to the block I/O layer 204 and emptied by the block drivers. Accordingly, a window slice refers to capturing a set of I/O requests in a time window that has sufficient I/O activity via use of a threshold (at 312, 314). To provide adaptiveness to the method 300, throughput of the shared data storage area, for example, the shared disk 210, is also periodically checked to see if it is degraded due to the execution of the software programming for the method 300 (at 316). The key idea of the method 300 is to obtain a window slice that will last long enough in the host kernel to finish the coordination; otherwise, the requests may be already issued at the host system to the disk before the manageability system 230 finishes its coordination and nullifies any benefits (at 320-322). Once the window slice is obtained, it is captured in an appropriate format and sent to the manageability system 220 and its daemon (324), and the host daemon goes back to sleep (318).
The method 400 begins with the manageability daemon waking up or activated 410 upon receiving an event such as a dump of the host I/O queue (at 410). Once the dump is received at the block I/O layer 208 of the manageability system 230, the manageability system OS applies a reordering strategy for its own I/O queue. The reordering subsystem maintains an internal queue that combines the queue requests from the host and the manageability system (at 414) such that the requests from the manageability system follow that of the host system. This is illustrated by an example in
Given such a combined queue, the reorder subsystem in the manageability system 230 subsequently picks a certain threshold percentage of the queue, for example, 80% of the queue and calculates the ratio of host and manageability requests (at 418). If the ratio is very large, it implies there is not much overlap between the host and manageability I/O requests, in which case the strategy is to not reorder the manageability queue but rather throttle, i.e., delay, the manageability requests as much as possible (422). Throttling helps delay the manageability requests and thus potentially prevents the manageability system to thrash the host requests. If the host queue is very small leading to the ratio being close to 0, again it may not be worthwhile or needed to do any reordering, and in this case the manageability system may keep issuing I/O requests which are then delayed as much as possible (422). If the ratio is close to 1, then there is a case for reordering (at 420). In this case, the reorder subsystem removes the host system I/O requests from the combined queue and the residual queue becomes the manageability I/O queue as illustrated in
The file system coupling is provided for two functions. First, during the boot phase, the manageability system 230 requests the host system 220 to provide information about the file system type for the shared disk. Second, during the operation phase, the host and manageability systems coordinate to maintain file system consistency on the shared disk 210. To support the first function, the manageability system 230 is designed to have support for multiple file systems in the manageability kernel. For example, OS'es such as Linux provide such support and may be leverage in the design of the system 230. Given support for such heterogeneous file systems, the file system coupling design is as follows. A stub at the host OS in the host system 220 provides knowledge of the host file system type to the manageability system 230. This information is then used by the MP to choose the appropriate file system modules and the corresponding host file system is mounted at the manageability system.
To support the second function for the file system coupling, the host system 220 is set as the master and the sole entity responsible for all file system writes; whereas, the manageability system 230 mounts the file system read-only. For writes, the failed writes are intercepted and redirected to the host system. The host system 230 executes the writes. In this way, file consistency is maintained. If a host application performs any writes that are not written to disk while the manageability application has started, the writes will not be reflected at the manageability system 230. However, the next time the manageability application runs, those writes would be visible and hence they would be eventually considered by it.
As shown in
At the application layers 201 and 205, management task coupling is provided. For example, a system monitoring solution in many scenarios split across the host and proposed manageability system. Examples of monitoring applications at the host system are those for collecting application health and performance information, as well as OS level logging information. Monitoring applications at the manageability system include those for collecting power and disk activity information. The monitoring data collected by these monitoring applications are exchanged across the host and manageability systems using closely coupled interfaces. In another example, a system asset management solution includes inventory information related to applications and OS/VMs running on the host system are sent via closely coupled interfaces to the manageability system. In still another example, monitoring information collected at the host system is exchanged to the manageability system where a diagnostics engine analyzes the data. Subsequently, policy decisions made by the diagnostics/analysis engine are exchanged to the host system for actuation. Such exchange of information across the host & manageability system take place through closely coupled interfaces.
Additional heuristics related to CPU scheduling that may be considered for improving system performance. For example, the host daemon may provide periodic information of the CPU activity at the host system 220. If the host system is very heavily loaded, then it may be inferred that there is a potential that I/O requests are also being issued. Consequently, the manageability system 230 may decide to perform throttling and slow down its rate of requests. Also, management task specific coupling may be considered either for performance reasons or for better design functionality.
In recap, the systems and methods as described herein provide closed-coupling of operating system pairs and middleware stacks of two systems so as to better provide coordination between two software architectures when managing shared resources.
What has been described and illustrated herein is an embodiment along with some of its variations. The terms, descriptions and figures used herein are set forth by way of illustration only and are not meant as limitations. Those skilled in the art will recognize that many variations are possible within the spirit and scope of the subject matter, which is intended to be defined by the following claims—and their equivalents—in which all terms are meant in their broadest reasonable sense unless otherwise indicated.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5619726 | Seconi | Apr 1997 | A |
| 7334086 | Hass | Feb 2008 | B2 |
| 7424711 | Schluessler | Sep 2008 | B2 |
| 7694159 | Cepulis | Apr 2010 | B2 |
| 7702966 | Chandwani | Apr 2010 | B2 |
| 7757023 | Ranganathan | Jul 2010 | B1 |
| 7805734 | McKinley | Sep 2010 | B2 |
| 8176168 | Ramamurthy | May 2012 | B1 |
| 8185913 | Talwar | May 2012 | B1 |
| 20040255055 | Lamberts | Dec 2004 | A1 |
| 20050160423 | Bantz | Jul 2005 | A1 |
| 20060095551 | Leung et al. | May 2006 | A1 |
| 20060107311 | Dawson | May 2006 | A1 |
| 20060130060 | Anderson et al. | Jun 2006 | A1 |
| 20060184942 | Cox | Aug 2006 | A1 |
| 20060271931 | Harris et al. | Nov 2006 | A1 |
| 20070011491 | Govindarajan et al. | Jan 2007 | A1 |
| 20070067769 | Geisinger | Mar 2007 | A1 |
| 20070104452 | Wang | May 2007 | A1 |
| 20070198750 | Moilanen | Aug 2007 | A1 |
| 20080005748 | Mathew et al. | Jan 2008 | A1 |
| 20080104591 | McCrory et al. | May 2008 | A1 |
| 20080104601 | Kaneko et al. | May 2008 | A1 |
| 20080126581 | Komikado et al. | May 2008 | A1 |
| 20080154985 | Childs | Jun 2008 | A1 |
| 20080222309 | Shanbhogue | Sep 2008 | A1 |
| 20080222311 | Lee et al. | Sep 2008 | A1 |
| 20080250222 | Gokhale et al. | Oct 2008 | A1 |
| Entry |
|---|
| Office Actions of File History of U.S. Appl. No. 11/888,348, dated May 24, 2010 and Nov. 9, 2009 (23 pages). |
| Office Actions of File History of U.S. Appl. No. 11/888,343, dated Jan. 25, 2012, Jun. 9, 2011, and Dec. 21, 2010 (58 pages). |
