Patent Application
20150186256
Embodiments of the present disclosure relate to the field of storage, and more particularly, towards providing a virtual storage pool for a target application.
With the constant developments in the field of computer technology, newer storage techniques are increasingly finding importance. For example, software-Defined Storage (SDS) platforms (like EMC® ViPR™) provide capabilities to abstract physical storage arrays, so that physical storage capabilities can be abstracted into a couple of single pools of virtual storage for target applications. Based thereon, storage administrators then create one or more virtual storage pools (VSP) with respect to unique requirements of the various target applications. For example, a transactional workload would subscribe to a virtual storage pool that features the characteristics of a high-performance block store such as EMC® VMAX®. A cloud application such as an online and content sharing file would subscribe to a virtual storage pool that features the characteristics of a distributed object or file-based storage cloud.
In traditional VSP techniques, however, management of virtual storage pools (e.g. in ViPR) to a large extent relies on the storage administrator, thereby leading to an extremely low efficiency. In addition, the management of virtual storage pools in reliance on administrator operations is largely affected by the domain expertise, experience and even working status of the administrator.
Besides, the virtual storage pools that are provided are almost static in nature. Once created, a virtual storage pool's configuration and physical resource allocation usually remain unchanged throughout its lifecycle, which makes it impossible to efficiently and effectively react to incidents that can occur at runtime.
To this end, the present disclosure provides a technique for providing a virtual storage pool for a target application.
According to an embodiment of the present disclosure, there is provided an apparatus and a method for providing a virtual storage pool for a target application, which includes receiving performance requirements associated with the target application; and providing a virtual storage pool set for the target application according to the performance requirements and on the basis of storage capabilities of physical storage resources, the virtual storage pool set comprising one or more virtual storage pools.
According to an optional embodiment of the present disclosure, the providing a virtual storage pool set for the target application according to the performance requirements and on the basis of storage capabilities of physical storage resources includes building a performance prediction model on the basis of historical data of one or more historical target applications, the historical data comprising a virtual storage pool set that is provided for each of the one or more historical target applications, as well as corresponding performance obtained; generating one or more alternative virtual storage pool sets for the target application according to the performance requirements and on the basis of the storage capabilities of the physical storage resources; respectively predicting performance of the one or more alternative virtual storage pool sets by using the performance prediction model; and selecting, on the basis of the prediction, one alternative virtual storage pool set from the one or more alternative virtual storage pool sets so as to be provided to the target application.
According to an optional embodiment of the present disclosure, the method further includes monitoring status of one or more virtual storage pools in the virtual storage pool set; detecting runtime events related to one or more virtual storage pools in the virtual storage pool set; and adjusting, on the basis of the status and the runtime events, the virtual storage pool set provided for the target application.
According to an optional embodiment of the present disclosure, the status comprises one or more of: Quality of Service of one or more virtual storage pools in the virtual storage pool set for the target application, performance metrics of one or more virtual storage pools in the virtual storage pool set, physical resource utilization of one or more virtual storage pools in the virtual storage pool set, interactions and contentions between multiple virtual storage pools.
According to an optional embodiment of the present disclosure, the runtime events comprise changes of the target application and/or changes of the physical storage resources.
According to an optional embodiment of the present disclosure, the step of adjusting the virtual storage pool set provided for the target application includes creating a new virtual storage pool in the virtual storage pool set, reclaiming an inactive virtual storage pool in the virtual storage pool set, or migrating workloads between multiple virtual storage pools in the virtual storage pool set.
According to an optional embodiment of the present disclosure, the adjustment to the virtual storage pool set is further based on input from a user.
According to an optional embodiment of the present disclosure, the performance requirements includes one or more of parameter such as capacity, cost, latency and a number of input/output operations per second (IOPs).
According to an embodiment of the present disclosure, the apparatus can includes one or more additional modules configured to execute the method steps discussed above, wherein the module may be part of the system, i.e., integrated into the system or may be a standalone module that can communicate with the apparatus to perform the method steps.
Note that the flowcharts and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which includes one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
With reference to several exemplary embodiments shown in the accompanying drawings, description is presented below to the principle and spirit of the present disclosure. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and further implement the present invention, rather than limiting the scope of the present invention in any manner.
Reference is made to
Next the method proceeds to step S120 where a VSP set is provided for the target application on the basis of storage capabilities of physical storage resources according to the performance requirements, wherein the VSP set includes one or more VSPs. It should be understood that the virtual storage technique is built on the basis of physical storage resources like physical storage arrays, so to provide a VSP set for the target application, and storage capabilities of various physical storage arrays should be taken into consideration. In practice, since these storage capabilities are usually associated with various storage arrays in the form of technical specification (usually machine-readable), they can be obtained easily.
Optionally, a mapping relationship between the target application and all VSPs in the provided VSP set may be recorded; optionally, a mapping relationship between one or more VSPs involved in the provided VSP set and corresponding physical storage resources may be recorded for purposes of the follow-up, lookup, reference and so on.
In addition, the method further includes an optional step: receiving the user's input as adjustment to the VSP set provided for the target application (not shown in the figure). This indicates that the method for providing VSP for a target application according to the embodiment of the present disclosure can take users' demands or recommendations into consideration wherever it is necessary.
The method for providing VSP for a target application depicted with reference to
According to one optional embodiment of the present disclosure, step S120 as shown in
The method then proceeds to step S1202 where one or more alternative virtual storage pool sets are generated for the target application on the basis of storage capabilities of the physical storage resources according to the performance requirements. It should be understood that there could exist a variety of VSP set configurations for selection with respect to a specific target application according to specific performance requirements and storage capabilities of physical storage resources. In this specification, possible various VSP sets are referred to as alternative VSP sets in order to distinguish from the VSP set that is finally provided to the target application.
Next in step S1203, performance of the one or more alternative virtual storage pool sets is predicted using the performance prediction model, respectively; and in step S1204, one alternative virtual storage pool set is selected from the one or more alternative virtual storage pool sets in order to be provided to the target application, on the basis of the prediction. For example, the optimal set of initial VSPs may be found by enumerating (with certain algorithms like dynamic programming, linear programming and Hill-Climb) different alternative VSP set and configuration combinations, and then comparing their performance utilizing the performance prediction model.
With reference to
1) changes from application workloads, e.g. workload shift and SLO shift;
2) VSP resource contention (e.g. network bandwidth, memory and cache) with other VSPs;
3) VSP storage resource shortage or underutilization;
4) underlying physical storage resource changes, e.g. storage array failures.
In response to these above mentioned incidents, the method for managing VSP according to the embodiment of the present disclosure will dynamically adjust the set and the configurations of active VSPs, as well as dynamically adjust the workload for each application of the VSP to be subscribed to. For example, when a new physical storage array joins, it will be immediately allocated to an existing or newly created virtual storage pool; when a virtual storage pool is short of physical resources to serve its application workloads, and it will obtain additional resources from other virtual storage pools whose loads are low, or some of its application workloads may be migrated to other virtual storage pools. Such a runtime VSP adjustment and workload migration procedure may be conducted as illustrated by
In step S130, status of one or more virtual storage pools in the virtual storage pool set is monitored. At runtime, statuses of VSPs that are live (alive) will be monitored for information like the QoS of VSPs for subscribing workloads, the VSP performance metrics, the physical resource utilization of VSPs, the interactions and contentions between VSPs.
In step S140, runtime events related to one or more virtual storage pools in the virtual storage pool set are detected. The runtime events include, for example, changes of the target application and/or changes of physical storage resources. Specifically, at runtime, various system incidents relevant to VSPs can be detected and reported, especially changes of the running workload set and underlying physical (storage) resources, e.g. newly joined physical storage arrays/pools.
Then in step S150, the virtual storage pool set provided for the target application is adjusted based on the status and runtime events. Specifically, at runtime, physical resources will be dynamically scheduled and reallocated among VSPs, new and deactivate deprecated VSPs created, and workloads migrated between different VSPs.
In an optional implementation, both the VSP status monitoring and runtime event detection may be continuous, i.e., runtime VSP status and runtime event information may be continuously collected. As another alternative, information on the VSP status may be, for example, periodically pushed for adjustment, and information on runtime events is preferably pushed as soon as it is detected, so as to learn in real time information such as changes involving hardware resources.
Also note that any adjustment to the VSP set can consider the user's suggestions/inputs, i.e., based on an adjustment made by a user, to control VSPs provided for the target application more flexibly. The user, therefore, will make adjustments on these VSPs and/or their detailed physical resource allocations and configurations, according to his knowledge, practices or other demands.
In addition, those skilled in the art should understand that during implementation, step S150 may be, by means of example, implemented via the step of making an adjustment decision on the basis of the monitored status and the monitored runtime events and the step of execution according to the made adjustment decision. In this case, after the adjustment decision is made, input from the user may be received, the adjustment decision is altered according to the input from the user, and then the altered adjustment decision is executed. The user may alter a decision that has previously been made on the basis of the VSP status and runtime events. However, those skilled in the art should further understand this implementation is merely for illustration; and in practice, step S150 may be completed using either more or less number of steps, and receiving input from the user may be implemented at any moment before making an adjustment. Various practices are only intended for a discussion on the method's implementation sphere and are not limiting the spirit and essence of the present disclosure.
In general, the method shown with reference to
According to an optional embodiment of the present disclosure, virtual storage pool providing module 402 further includes model building module 4021 configured to build a performance prediction model on the basis of historical data of one or more historical target applications, the historical data comprising a virtual storage pool set that is provided for each of the one or more historical target applications, as well as corresponding performance obtained; alternative generating module 4022 configured to generate one or more alternative virtual storage pool sets for the target application according to the performance requirements and on the basis of the storage capabilities of the physical storage resources; predicting module 4023 configured to respectively predict performance of the one or more alternative virtual storage pool sets by using the performance prediction model; and selecting module 4024 configured to select, on the basis of the prediction, one alternative virtual storage pool set from the one or more alternative virtual storage pool sets so as to be provided to the target application. Each of these sub-modules model building model 4021, alternative generating module 4022, predicting module 4023 and selecting module 4024, can be combined into the optimization module 400, wherein the optimization module is configured to perform the tasks of these sub-modules.
According to an optional embodiment of the present disclosure, system 400 further includes monitoring module 403 configured to monitor status of one or more virtual storage pools in the virtual storage pool set; detecting module 404 configured to detect runtime events related to one or more virtual storage pools in the virtual storage pool set; and adjusting module 405 configured to adjust, on the basis of the status and the runtime events, the virtual storage pool set provided for the target application. Again monitoring module 403 and detecting module 404, can be combined into the optimization module 400, wherein the optimization module is configured to perform the tasks of these modules.
According to an optional embodiment of the present disclosure, the status comprises one or more of the following parameters: Quality of Service for one or more virtual storage pools in the virtual storage pool set for the target application, performance metrics for one or more virtual storage pools in the virtual storage pool set, physical resource utilization for one or more virtual storage pools in the virtual storage pool set, interactions and contentions between multiple virtual storage pools.
According to an optional embodiment of the present disclosure, the runtime events comprise changes of the target application and/or changes of the physical storage resources.
According to an optional embodiment of the present disclosure, the adjusting module (or the optimization module) adjusting the virtual storage pool set provided for the target application includes creating a new virtual storage pool in the virtual storage pool set, reclaiming an inactive virtual storage pool in the virtual storage pool set, or migrating workloads between multiple virtual storage pools in the virtual storage pool set.
According to an optional embodiment of the present disclosure, the adjustment made by adjusting module 405 to the virtual storage pool set is further based on input from a user. Adjusting Module 405, can be combined into the optimization module 400, wherein the optimization module is configured to perform the tasks of these modules.
According to an optional embodiment of the present disclosure, the performance requirements comprise one or more of the following parameters: capacity, cost, latency and input/output operations per second (IOPs).
Those skilled in the art may understand that the system for providing a virtual storage pool set for a target application as illustrated in
It should be understood that the structural block diagram in
In particular, besides hardware embodiments, the embodiments of the present disclosure may also be implemented in a form of a computer program product. For example, the method as described with reference to
Illustration has been presented above to the spirit and principle of the present disclosure with reference to several concrete embodiments. According to the embodiment of the present disclosure, a logical storage block of appropriate size may be selected for a file according to the file's characteristics, and an on-demand physical storage area allocation may be adopted, so that fragments can be reduced without increasing metadata and location overhead and further a flexible and efficient management of file systems can be achieved.
It should be noted that, the embodiments of the present disclosure can be implemented in software, hardware or the combination thereof. The hardware part can be implemented by a dedicated logic; the software part can be stored in a memory and executed by a proper instruction execution system such as a microprocessor or a design-specific hardware. One of ordinary skill in the art may understand that the above-mentioned method and system may be implemented with a computer-executable instruction and/or in a processor controlled code, for example, such code is provided on a bearer medium such as a magnetic disk, CD, or DVD-ROM, or a programmable memory such as a read-only memory (firmware) or a data bearer such as an optical or electronic signal bearer. The apparatuses and their modules in the present invention may be implemented by hardware circuitry of a very large scale integrated circuit or gate array, a semiconductor such as logical chip or transistor, or a programmable hardware device such as a field-programmable gate array or a programmable logical device, or implemented by software executed by various kinds of processors, or implemented by combination of the above hardware circuitry and software such as firmware.
It should be noted that although a plurality of units or subunits of the apparatuses have been mentioned in the above detailed depiction, such partitioning is merely non-compulsory. In actuality, according to the embodiments of the present disclosure, the features and functions of two or more units above described may be embodied in one unit. On the contrary, the features and functions of one unit above described may be further partitioned to be embodied in more units.
In addition, although operations of the method of the present disclosure are described in specific order, it should not be construed as requiring or suggesting such operations be completed in the shown specific order, or as requiring all depicted operations to be executed for achieving desired results. On the contrary, the steps depicted in the flowchart may change execution order. Additionally or alternatively, some steps may be omitted, multiple steps may be combined to one step for execution, and/or one step may be decomposed into multiple steps for execution.
Although the present disclosure has been depicted with reference to a plurality of embodiments, it should be understood that the present disclosure is not limited to the embodiments disclosed herein. On the contrary, the present disclosure intends to cover various modifications and equivalent arrangements included in the spirit and scope of the appended claims. The scope of the appended claims meets the broadest explanations and covers all such modifications and equivalent structures and functions.
| Number | Date | Country | Kind |
|---|---|---|---|
| 201310755994.3 | Dec 2013 | CN | national |
This Application claims priority from Provisional Application Ser. No. CN201310775994.3 filed on Dec. 27, 2013 entitled “METHOD AND SYSTEM FOR PROVIDING VIRTUAL STORAGE POOLS TO TARGET APPLICATIONS,” the content and teachings of which are hereby incorporated by reference in their entirety.
