Patent Application
20230071976
This application claims the benefit of Taiwan application Serial No. 110133368, filed Sep. 8, 2021, the subject matter of which is incorporated herein by reference.
The present disclosure relates in general to a virtual function performance analysis system and an analysis method thereof, and relates to an analysis system for execution performance of virtual function in a virtual platform infrastructure and an analysis method thereof.
Along with the evolution of network communication infrastructure and software technology, each physical computing resource or each storage resource may be virtually operated on a virtual platform infrastructure. On the virtualized infrastructure, a terminal application of a user (i.e., a virtual network function application) may flexibly access the virtual computing resources or storage resources in the virtual platform infrastructure.
However, the execution of the virtual network function application and the access of the virtual resources in an environment of virtual platform are far more complicated than that in a physical environment. Moreover, the virtual network function application may be a free of charge open-source software or a package provided by a third party. If the virtual network function application has abnormalities, overall system performance on the virtual platform will deteriorate. However, it is very difficult for the user to identify which functional component (e.g., virtual function) of the virtual network function application has abnormalities.
Therefore, research and development personnel in relevant industries of the present technology field are engaged in analyzing performance of internal functional components (i.e., virtual functions as programmatic function calls) of the virtual network function application with an aim to more accurately analyze what factors deteriorate system performance on the virtual platform and to identify the bottleneck of system performance on the virtual platform.
According to one embodiment of the present disclosure, a virtual function performance analysis system is disclosed. The virtual function performance analysis system includes a monitoring unit, a performance analysis unit and a resource adjusting unit. The monitoring unit is configured to monitor performance of at least one virtual function of a virtual network function application on a virtual platform, wherein the virtual platform has at least one physical resource and at least one virtual resource, each physical resource, each virtual resource and each virtual function is evaluated with at least one performance indicator respectively; each of the performance indicators is associated with an expected value and/or a threshold value for each target virtual resource or virtual function; the monitoring unit monitors and records an actual value of each of the performance indicators on the virtual platform. The performance analysis unit is configured to compare the actual value of each of the performance indicators with the associated expected value and/or threshold value to obtain a comparison result of each of the performance indicators and to analyze system performance according to the comparison result.
According to another embodiment of the present disclosure, a virtual function performance analysis method is disclosed. The virtual function performance analysis method includes: monitoring performance of at least one virtual function of a virtual network function application on a virtual platform, wherein the virtual platform has at least one physical resource and at least one virtual resource, each physical resource, each virtual resource and each virtual function is evaluated with at least one performance indicator respectively; each of the performance indicators is associated with an expected value and/or a threshold value respectively. Monitoring and recording an actual value of each performance indicator on the virtual platform. Comparing the actual value of each performance indicator with the associated expected value and/or threshold value to obtain a comparison result of each of the performance indicators and analyzing system performance according to the comparison result.
The above and other aspects of the invention will become better understood with regard to the following detailed description of the preferred but non-limiting embodiment (s). The following description is made with reference to the accompanying drawings.
Technical terms are used in the specification with reference to generally-known terminologies used in the technology field. For any terms described or defined in the specification, the descriptions and definitions in the specification shall prevail. Each embodiment of the present disclosure has one or more technical features. Given that each embodiment is implementable, a person ordinarily skilled in the art may selectively implement or combine some or all of the technical features of any embodiment of the present disclosure.
On the virtual platform of an actual environment where the virtual network function application is executed, the monitoring unit 104 may monitor performance of at least one virtual function of the virtual network function application. More specifically, the virtual network function application and each virtual function respectively are evaluated with one or more than one performance indicator; each performance indicator may reflect performance of the virtual network function application and each virtual function executed on the virtual platform. For a target virtual function, each performance indicator is associated with an expected value, which may be system default or user defined. Besides, long-term status of each virtual function operated on the virtual platform may be observed and analyzed through machine learning to create an expected value associated with each performance indicator. Moreover, an usage status of each physical resource and/or each virtual resource used by the virtual network function application on the virtual platform may also serve as a performance indicator of the virtual function performance analysis system 100A.
The monitoring unit 104 monitors the execution of each virtual function and accordingly records actual performance of each virtual function on the virtual platform to obtain an actual value of each performance indicator with which each virtual function is evaluated. Moreover, the monitoring unit 104 also monitors the status of each virtual resource and/or each physical resource used by each virtual network function application on the virtual platform and accordingly records the actual value of each performance indicator with which each virtual resource and/or each physical resource is evaluated.
The performance analysis unit 106 analyzes and compares the actual value of each performance indicator of each virtual resource and/or each physical resource and each virtual function with associated expected values, analyzes and compares the status of each virtual resource and/or each physical resource used by each virtual network function application to obtain a comparison result, and analyzes system performance according to the comparison result, so as to determine whether the virtual network function application and each virtual function have abnormalities which deteriorate overall system performance on the virtual platform. The performance analysis unit 106 further precisely locates particular virtual function, virtual resource or physical resource, which deteriorates performance, and identifies possible abnormalities leading to bottleneck of system performance.
Based on the identified abnormalities, the resource adjusting unit 108 may perform system resource adjustment according to the execution status of the virtual network function application or each virtual function, for example, by expanding service of the virtual network function application to reduce the workload of the virtual function that causes performance bottleneck, Or, the resource adjusting unit 108 may adjust the allocation of physical resources and/or virtual resources on the virtual platform, for example, by expanding the allocation of physical resources and/or virtual resources to assure that sufficient physical resources and/or virtual resources may be allocated to the virtual network function application which has the virtual function as a bottleneck.
In other words, unlike the expected value associated with each performance indicator in the virtual function performance analysis system 100A which may be system default or user defined, the expected value associated with each performance indicator in the virtual function performance analysis system 100B is created by the testing unit 102 on the testing platforming or through machine learning beforehand.
Descriptions of the virtual function performance analysis system 100A or 100B of the present disclosure are disclosed above with accompanying drawings
Firstly, let the performance indicator “execution time” of each of the virtual functions 202, 204 and 206 be taken for example. The execution time directly reflects the performance of each of the virtual functions 202, 204 and 206. If the execution time is too long, this indicates that one of the following scenarios may occur: the execution of each of the virtual functions 202, 204 and 206 is blocked; the workload of each of the virtual functions 202, 204 and 206 is overloaded; the physical resources are competed on the virtual platform; or the physical resources and/or virtual resources allocated to the virtual network function application 200 (which has the virtual functions 202, 204 and 206) are insufficient.
More specifically, taking the virtual function 202 as an example, the execution time T_02 of the virtual function 202 is the time interval between the starting time point t1 and the end time point t2 of the execution of the virtual function 202. The starting time point t1 may be the time point when the virtual function 202 is called; the end time point t2 may be the time point when the virtual function 202 returns the result. In another implementation, if the virtual function 202 may be executed in a multi-thread environment, then the execution time T_02 of the virtual function 202 is the respective execution time of the virtual function 202 corresponding to each thread of multi-thread.
As disclosed above, in the virtual function performance analysis system 100B of
More specifically, let the execution time T_02 of the virtual function 202 be taken for example. The testing unit 102 may observe the execution time T_02 using eBPF. Or, in another implementation, when the user programs the programming code of the virtual function 202, observation points may be created in the programming code of the virtual function 202 to record the time points (starting time point t1 and end time point t2) at which the virtual function 202 is called and returns the result respectively, such that the execution time T_02 may be calculated according to the log of the observation points created in the programming code.
Under different environmental scenarios, the execution time T_02, T_04 and T_06 of each of the virtual functions 202, 204 and 206 may vary Therefore, the testing unit 102 may collect the statistics of the execution time T_02, T_04 and T_06 of each of the virtual functions 202, 204 and 206 under different environmental scenarios to obtain the minimum value, mean value and maximum value of the execution time T_02, T_04, T06. For example, as listed in Table 1, on the testing platform 300, the expected values of the execution time T_02, T_04 and T_06 of each of the virtual functions 202, 204 and 206 built by the testing unit 102 are as follows. For the expected value of the execution time T_02 of the virtual function 202, the minimum is 50 us, the mean is 100 us, and the maximum is 200 us. For the expected value of the execution time T_04 of the virtual function 204 the minimum is 30 us, the mean is 50 us, and the maximum is 150 us. Moreover, for the expected value of the execution time T_06 of the virtual function 206, the minimum is 100 us, the mean is 130 us, and the maximum is 250 us.
On the other hand, refer to
The virtual function performance analysis system 100A (or 100B) may set a threshold value for the queue length of the queue 202a and queue 204a respectively, wherein the threshold value may be system defaulted or user defined. For example, the user (virtual function developer) may collect statistics and evaluate a reasonable queue length for the queue 202a and the queue 204 respectively according to the historical data of use, and set the reasonable queue length as a threshold value associated with the performance indicator “queue length”. As listed in Table 1, the threshold value of the queue length of the queue 202a at the input end of the virtual function 202 may be set to 200. When the virtual function 202 is executed, if the queue length of the queue 202a is over the threshold value, this indicates the virtual function 202 has abnormalities. On the other hand, for the queue 204a at the output end of the virtual function 204, if the queue length of the queue 204a is too long, this indicates that the other virtual function 302 using the queue 204a as the input end possibly may have abnormalities.
Moreover, each of the virtual functions 202, 204 and 206 may be evaluated with other performance indicators, such as “execution frequency” and “error rate”, wherein the “execution frequency” of each of the virtual functions 202, 204 and 206 is the number of times for which each of the virtual functions 202, 204 and 206 is executed within a unit time (i.e., number of executions being performed per unit time). Besides, if each of the virtual functions 202, 204 and 206 is executed in a multi-thread environment, the performance indicators “queue length”, “execution frequency” and “error rate” respectively are the queue length, execution frequency and error rate of each of the virtual functions 202, 204 and 206 corresponding to each thread of multi-thread.
The expected value or threshold value associated with each performance indicator of each of the virtual functions 202, 204 and 206 may be estimated by the testing unit 102 on the testing platform 300 through test or evaluation, may be user defined, may be system default, or may be created. For example, the actual value of each performance indicator of each of the virtual functions 202, 204 and 206 may be recorded over a long period of operation time, and the expected value or threshold value associated with each performance indicator may be obtained by statistical analysis through machine learning.
Except for the performance indicators “execution time”, “queue length”, “execution frequency” and “error rate”, the “usage status” of physical resources and virtual resources of virtual platform used by the virtual network function application 200 (which has the virtual functions 202, 204 and 206) also reflects performance of the virtual network function application 200. Therefore, the “usage status” of the physical resources and/or virtual resources used by the virtual network function application 200 is a performance indicator which the virtual function performance analysis system 100A (or 100B) would like to analyze.
On the other hand, on a virtual platform 400, the physical resources may be virtualized as corresponding virtual resources on a virtual layer 410, The physical CPU 402, the physical memory 404, the physical network bandwidth 406, and the physical disk input/output 408 respectively correspond to a virtual CPU 412, a virtual memory 414, a virtual network bandwidth 416, and a virtual disk input/output 418. Moreover, the virtual platform 400 may allocate the virtual resources to the virtual network function application 200 and other virtual network function applications. Also referring to
Referring to Table 2, several performance indicators of the “usage status” of each physical resource and/or each virtual resource on the virtual platform 400 are listed. In a broad sense, the performance indicator with which each physical resource and/or each virtual resource is evaluated may comprise the “usage status” of each physical resource and/or each virtual resource. More particularly, the performance indicator type related to the “usage status” of each physical resource and/or each virtual resource may include “value of usage status”, “error event” and “resource saturation” of each physical resource and/or each virtual resource. Let the type of performance indicator be “value of usage status”. Details of the performance indicator “value of usage status” of each physical resource and/or each virtual resource may include value of usage status of single CPU, memory and disk respectively and average value of usage status of the overall system, and transmitted/received data flow and bandwidth of a network interface.
Based on the performance indicators listed in Table 2, the virtual function performance analysis system 100A (or 100B) may create the expected value and/or threshold value associated with each performance indicator according to system default value or user defined value. Refer to Table 2-1. The virtual network function application 200 may set the expected value of the “value of usage status” of the virtual resource 412a of the virtual CPU 412 to 22%; the virtual network function application 230 may set the expected value of the “value of usage status” of the virtual resource 412b of the virtual CPU 412 to 25%; the virtual network function application 260 may set the expected value of the “value of usage status” of the virtual resource 412c of the virtual CPU 412 to 15%. Moreover, the virtual network function applications 200, 230 and 260 may set the expected value of the total value of usage status of the physical CPU 402 to 62%.
As disclosed above, the monitoring unit 104 also monitors and records actual value of usage status of each virtual resource and/or each physical resource used by the virtual network function application 200 (that is, target virtual network function application) and other virtual network function applications 230 and 260 on the virtual platform 400 to obtain actual value of each performance indicator of each physical resource and/or each virtual resource. Let performance indicator “value of usage status” be taken for example. Actual values of the values of usage status of the virtual CPU 412 and/or physical CPU 402 of the virtual network function applications 200, 230 and 260 recorded by the monitoring unit 104 are listed in Table 4.
As listed in Table 4, actual value of usage status of the virtual CPU 412 (virtual resource 412a) set on or allocated to the virtual network function application 200 is 20%; actual value of usage status of the virtual CPU 412 (virtual resource 412b) set on or allocated to the virtual network function application 230 is 22%; actual value of usage status of the virtual CPU 412 (virtual resource 412c) set on or allocated to the virtual network function application 260 is 18%. Besides, actual total value of usage status of the physical CPU 402 of the virtual network function applications 200, 230 and 260 is 60%.
To summarize, the monitoring unit 104 may monitor and record actual value of each performance indicator (such as “execution time”, “queue length”, “execution frequency”, or “error rate”) of each of the virtual functions 202, 204 and 206 of the virtual network function application 200 (target virtual network function application). Moreover, the monitoring unit 104 also may monitor and record the usage status of each physical resource and/or each virtual resource used by the virtual network function application 200 and other virtual network function applications 230 and 260 on the virtual platform 400 to record actual value of each performance indicator (such as “value of usage status”, “error event”, or “resource saturation”) of each physical resource and/or each virtual resource.
Thus, the performance analysis unit 106 may compare actual value of each performance indicator of each of the virtual functions 202, 204 and 206 of the virtual network function application 200 and actual value of each performance indicator of each physical resource and/or each virtual resource on the virtual platform 400 with associated expected value and/or threshold value to determine whether the virtual functions 202, 204 and 206 of the virtual network function application 200 have abnormalities or whether the usage status of each physical resource/each virtual resource is abnormal, so as to analyze performance bottleneck which causes abnormalities. Then, the resource adjusting unit 108 may adjust service of the virtual network function application 200 or the virtual functions 202, 204 and 206, adjust the virtual resources allocated to the virtual network function application 200 and/or other virtual network function applications 230 and 260, or expand the physical resources on the virtual platform 400. Detailed operations of the performance analysis unit 106 and the resource adjusting unit 108 are disclosed below with embodiments of the virtual function performance analysis method.
Then, the method proceeds to step 606, the virtual functions 202, 204 and 206 of the virtual network function application 200 are monitored in an actual environment on the virtual platform 400, wherein the virtual network function application 200 uses each virtual resource and each physical resource on the virtual platform 400.
Then, the method proceeds to step 608, actual value of each performance indicator of each of the virtual functions 202, 204 and 206 is monitored and recorded by the monitoring unit 104 of the virtual function performance analysis system 100A (or 100B) on the virtual platform 400.
Then, the method proceeds to step 610, actual condition of use (such as “value of usage status”) of each physical resource and each virtual resource by the virtual network function application 200 and other virtual network function applications 230 and 260 are monitored and recorded by the monitoring unit 104 on the virtual platform 400 to build actual value of each performance indicator of each physical resource and each virtual resource. Then, step 613 of
In the method 600 of
Refer to
For example, in step 614, the comparison result between the actual value and the expected value and/or threshold value associated with performance indicator indicates that the actual value of the performance indicator “execution time” of each of the virtual functions 202, 204 and 206 basically meets the expected value, wherein the actual value is not far greater than the expected value. As listed in Table 3, the actual value (includes the minimum, the mean and the maximum) of the execution time of each of the virtual functions 202, 204 and 206 basically meets the expected value of the execution time as listed in Table 1, this indicates that the actual value of the execution time of each of the virtual functions 202, 204 and 206 on a virtual platform 400 basically meets the expected value. Therefore, it is determined that the workload of each of the virtual functions 202, 204 and 206 is not overloaded. Meanwhile, step 616 may be performed to analyze whether actual values of other performance indicators are over the expected value and/or threshold value.
In step 616, the actual value of another performance indicator “queue length” is compared with the threshold value by the performance analysis unit 106. The comparison result obtained in step 616 indicates that the actual value of “queue length” is far greater than the threshold value (as listed in Table 3, actual value of the queue length of the virtual function 202 is 250, which is far greater than the threshold value of the queue length listed in Table 1, that is, 200). Based on the comparison result, it may be confirmed that the virtual function 202 has abnormalities, and performance bottleneck on the virtual platform 400 may be defined as the virtual function 202 whose actual value of queue length is far greater than the threshold value.
Then, the method proceeds to step 618, the workload of the virtual function 202 whose queue length is greater than the threshold value is determined as being overloaded by the performance analysis unit 106, and the resource adjusting unit 108 may adopt the following solutions to resolve the abnormalities of the virtual function 202, for example, by adjusting service of the virtual network function application 200 or by adjusting the allocation of the physical resources and/or virtual resources on the virtual platform 400.
In the present embodiment, the resource adjusting unit 108 adjusts service of the virtual network function application 200 by expanding other virtual network function applications similar to the virtual network function application 200 to share the workload of the virtual function 202 of the virtual network function application 200.
Refer to
Possibly, the virtual function 204 becomes bottleneck due to the usage status of the physical resources and/or virtual resources being near an upper limit of the allocation of corresponding resources or almost overload. For example, the physical resources and/or virtual resources allocated to the virtual network function application 200 (which has the virtual function 204) are insufficient, making the usage status almost overloaded. Or, in a multi-thread environment, other virtual functions 202 or 206 of the virtual network function application 200 compete with the virtual function 204 for virtual resources. Or, other virtual network function applications 230 and 260 compete with the virtual network function application 200 (target virtual network function application) for physical resources. Meanwhile, step 620 may be performed to further clarify the causes,
In step 620, whether the usage status of each physical resource and/or each virtual resource is near an upper limit of the allocation of corresponding resources or almost overloaded is determined by the performance analysis unit 106 according to the recorded actual value of each performance indicator of each physical resource and/or each virtual resource. The analysis result is listed in Table 6. The actual value of usage status of the virtual CPU 412 used by the virtual network function application 200 (that is, target virtual network function application, which has the virtual function 204) is 54%, being far greater than the expected value as listed in Table 2-1, that is, 20%. This indicates that the usage status of the virtual CPU 412 used by the virtual network function application 200 is already near an upper limit of the allocation of the virtual resource 412a allocated to the virtual network function application 200, or the usage status is near the upper limit of the allocation or almost overloaded. Therefore, performance bottleneck of the virtual function 204 could possibly occur due to the following: resource (i.e., virtual resource 412a) of the virtual CPU 412, which is allocated by the virtual platform 400 to the virtual network function application 200 (which has the virtual function 204), is insufficient.
Then, the method proceeds to step 622, the allocation of each virtual resource of the network function application 200 is adjusted by the resource adjusting unit 108. For example, the allocation of the virtual CPU 412 may be adjusted, that is, more virtual resource 412a of the virtual CPU 412 is allocated to the virtual network function application 200. Or, the resource adjusting unit 108 may expand the physical resources and/or virtual resources on the virtual platform 400. Or, the resource adjusting unit 108 may adjust service of the virtual network function application 200, for example, by expanding to execute other virtual network function applications with the same functions of the virtual network function application 200 to share the workload of the virtual network function application 200.
Refer to
Next, step 624 is performed to further analyze whether the total value of usage status of the physical resources used by the virtual network function applications 200, 230 and 260 on the virtual platform 400 is near the upper limit of the allocation of the physical resources or almost overloaded.
In step 624, the analysis result recorded by the performance analysis unit 106 as listed in Table 7 shows that the actual value of total value of usage status (i.e., total actual value of usage status) of the physical CPU 402, that is, 95%, is near an upper limit of the allocation of the physical CPU 402 and almost overloaded, therefore it is determined that the virtual network function applications 230 and 260 other than the virtual network function application 200 are competing for the physical CPU 402, and the value of usage status of the physical CPU 402 used by the other virtual network function applications 230 and 260 is too high and results in overloading. As listed in Table 7, the value of usage status of the resource of the virtual CPU used by the virtual network function application 230 is 52% (52% is too high), this indicates the virtual network function application 230 is competing for the resource of the physical CPU 402.
Then, the method proceeds to step 626, the allocation of each virtual resource of the virtual network function application 200 is adjusted by the resource adjusting unit 108. For example, the allocation of the virtual CPU 412 may be adjusted, that is, more virtual resource 412a of the virtual CPU 412 is allocated to the virtual network function application 200, such that the virtual network function application 200 has sufficient virtual resource allocations. Or, the resource adjusting unit 108 may expand the physical resources and/or virtual resources on the virtual platform 400. Or, the resource adjusting unit 108 may adjust the physical CPU 402 whose usage status is near an upper limit of the allocation of the physical CPU 402 or almost overloaded, for example, by expanding the physical resources and/or virtual resources on the virtual platform 400, such that more physical CPU may be provided.
Refer to
Then, the method proceeds to step 628, the performance analysis unit 106 determines that performance bottleneck of the virtual function 204 is possibly caused by logic error of programming in the programming code of the virtual function 204 (such as error in the lock mechanism of the programming code of the virtual function 204) or caused by abnormalities in packet processing. A case for the abnormality in packet processing may be, for example, packet loss of the virtual function 204 will cause the re-transmission mechanism of the software protocol to re-transmit the packets, and the delay in service of the virtual function 204 will be too long.
According to the embodiments of different implementations disclosed above, the virtual function performance analysis system 100, (or 100B) of the present disclosure monitors several performance indicators (such as “execution time”, “execution frequency”, “queue length and “error rate”) of each of the virtual functions 202, 204 and 206 of the virtual network function application 200 (that is, target virtual network function application) and analyzes several performance indicators (such as “value of usage status”, “error event” and “resource saturation”) of each physical resource and/or each virtual resource on the virtual platform 400. Moreover, actual value of each performance indicator of each physical resource and/or each virtual resource of each of the virtual functions 202, 204 and 206 of the virtual network function application 200 is monitored on the virtual platform 400 of an actual environment. Then, whether the actual value of each performance indicator is far greater than the expected value and/or threshold value is determined. If the actual value of each performance indicator is far greater than the associated expected value and/or threshold value, this indicates: the virtual function (such as virtual function 202) is overloaded; the physical resources and/or virtual resources allocated to the virtual network function application 200 (which has the virtual function that is overloaded) are insufficient; other virtual network function applications 230 and 260 are competing with the virtual network function application 200 for physical resources; or the virtual functions 202, 204 and 206 of the virtual network function application 200 are completing for virtual resources. The virtual function performance analysis system 100A (or 100B) automatically defines the “virtual function”, or the “physical resource” or “virtual resource” whose usage status is near the upper limit of the allocation of corresponding resources or almost overloaded as a bottleneck which deteriorates performance of the virtual network function application 200.
In comparison to the art in which the data of each performance indicator is manually collected by maintenance personnel on the virtual platform, the virtual function performance analysis system 100A (or 100B) of the present disclosure connects and analyzes the data of each performance indicator in an automatic manner and therefore has the advantage in terms of timeliness and speed. In the art, maintenance personnel on the virtual platform do not analyze the performance indicator “execution time” of the virtual function, and therefore may not accurately analyze the causes of abnormalities. Conversely, the virtual function performance analysis system 100A (or 100B) of the disclosure may accurately analyze the execution time of each virtual function. In comparison to the art which merely ascribes performance bottleneck to the virtual network function application, the virtual function performance analysis system 100A (or 100B) of the present disclosure may more accurately ascribe bottleneck to a particular function of the virtual network function application and accurately analyze which function deteriorates system performance on the virtual platform 400.
Besides, with respect to the factor that deteriorates system performance on the virtual platform, the virtual function performance analysis system 100, (or 100B) of the present disclosure may automatically adjust the virtual network function application 200 and/or service of each of the virtual functions 202, 204 and 206 or automatically expand the physical resources and/or virtual resources or adjust the allocation on the virtual platform 400 to improve system performance on the virtual platform 400 in an automatic manner.
While the invention has been described by way of example and in terms of the preferred embodiment (s), it is to be understood that the invention is not limited thereto. On the contrary, it is intended to cover various modifications and similar arrangements and procedures, and the scope of the appended claims therefore should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements and procedures.
| Number | Date | Country | Kind |
|---|---|---|---|
| 110133368 | Sep 2021 | TW | national |
