Patent Application
20230246903
The present disclosure relates generally to the monitoring of network traffic, in particular, of web traffic between a user and a web application. The disclosure relates specifically to the detection and evaluation of a network-induced quality of experience (QoE) degradation of the user. Thus, the disclosure provides a device and method for detecting a network-induced contribution to a determined QoE degradation of the user. To this end, the device may be configured to identify and disentangle the network-induced contribution to the determined QoE degradation from a content-induced contribution caused by a content change of the web application.
Inference of QoE of one or more users, and the detection of its degradation, is an important management tool for internet service providers (ISPs). Many objective in-browser metrics exist, and their monitoring from encrypted web traffic between the users and web applications is feasible. Thereby, the web traffic corresponds to the amount of data sent or received by the users to/from the web applications.
However, QoE degradations may be rooted at different sources. For instance, QoE degradations may be rooted in network-induced problems, i.e., problems related to the network that impacts the web traffic and thus causes the QoE degradations. Alternatively, the QoE degradations could be simply caused by a change of the content itself.
Some relevant studies in this field relate to the definition of QoE and its inference. In particular, some conventional techniques define web QoE metrics, which correlate with the QoE performance, so as to infer the web QoE metrics without having to access browser metrics. The QoE inference may be done from encrypted network traffic, which is what ISPs typically have access to.
Some other studies relate to web QoE monitoring. For instance, browser studies typically correlate a QoE change with changes in the software base.
Some other studies relate more specifically to QoE degradation, in particular, the detection of QoE degradations of users. The focus on QoE degradations from the perspective of the ISPs was proposed first for tackling general applications, then video applications, and more recently also web applications.
Some other studies relate to web content monitoring. Given that the web evolved much in the last thirty years, there is industrial effort to monitor its evolution, in particular, from the viewpoint of web page characteristics, or technologies, or content. Academia also regularly performs longitudinal surveys of content changes employing large datasets, but QoE is generally out of the picture in these studies.
The present disclosure bases on the fact that the conventional studies focus either on QoE or on content, but rarely are both aspects considered together. Furthermore, even when these aspects are considered together, only marginals are considered, but temporal aspects are not taken into account. That is, no information from content evolution is taken into account when evaluating QoE.
QoE degradations are conventionally identified, for instance, by linking network key performance indicator (KPI) features with application-specific metrics representative of user QoE (also called user KQI). However, this neglects the impact of content changes that regularly happen in web applications. For instance, when a web page of a web application changes, the KQI indicator can change as well (e.g., when the structure of the web page becomes more complex with more objects, it may take longer to download the web page, but this may not be related to any network limitation or to a reduced web traffic). The conventional methodology, which ignores content particularly content changes, is thus not reliable in the case of web applications, and would lead to many false alarms.
In view of the above-mentioned problems and disadvantages, embodiments of the present invention aim to improve the detection of QoE degradations. In particular, an objective is to enable a joint monitoring of QoE and web application content, in order to determine the network-induced contribution to a QoE degradation of a user, or of a set of users, of the web application.
The objective is achieved by the embodiments of the invention as described in the enclosed independent claims. Advantageous implementations of the embodiments of the invention are further defined in the dependent claims.
Embodiments of the invention provide a device and a method for detecting QoE degradations of users of web applications, wherein, instead of looking at the network level only, also content related information is considered. A device or method according to an embodiment of the invention may, to this end, employ active measurements for accessing the content related information. The embodiments of the invention base on solving two problems. Firstly, the problem of identifying and disentangling QoE degradations caused by network issues (for which ISPs are responsible) from apparent QoE degradations caused by content-based changes (for which ISPs are not responsible). Secondly, the problem of combining different network and content signals, in order to provide clues about the most probable origin of the determined QoE degradations.
A first aspect of the disclosure provides a device for detecting a network-induced contribution to a QoE degradation for one or more web applications, the device being configured to: obtain one or more user metrics, the user metrics being indicative of a QoE of one or more users of the web applications; obtain one or more content metrics, the content metrics being indicative of a content of the web applications; determine a QoE degradation of the QoE of the users based on the one or more user metrics; and determine a contribution information indicative of a network-induced contribution to the determined QoE degradation based on the content metrics.
The device of the first aspect is able to consider the content particularly content changes of the web applications, when detecting a QoE degradation of a user. For instance, the device may passively monitor the one or more user metrics (which may comprise one or more quality of service (QoS) metrics), and may further separately perform active measurements to track changes of the content for one or more selected web applications, i.e., to determine the content metrics.
The device may detect the QoE degradation following the user metrics. The one or more user metrics may be closely related to the QoE of the one or more users. The user metrics may be relevant to the one or more users (instead of purely network related metrics), and may be accessible (passively) even under encryption of the web traffic.
The device can further determine the network-induced QoE contribution to the QoE degradation. For instance, the device may distinguish, whether the determined QoE degradation is caused by a network-based issue or change and/or is caused by a content-based change. This can improve the detection and analysis of the determined QoE degradation. In particular, this may enable a reduction of false alarms. It may further be possible for the device to explain the determined QoE degradation, e.g., to determine the contribution to the QoE degradation of either content, network, or another source. This may allow for a better understanding of issues encountered by the users.
In an implementation form of the first aspect, the contribution information is indicative of one of: the network-induced contribution is the largest contribution to the determined QoE degradation; the network-induced contribution is not the largest contribution to the determined QoE degradation; the network-induced contribution is the only contribution to the determined QoE degradation; the network-induced contribution to the determined QoE degradation is zero.
The network-induced contribution is the contribution that the network (particularly, a network issue) has on the determined QoE degradation. The determined QoE degradation reflects the change of the QoE (and corresponding change of the user metrics) in a certain direction considered to indicate a worse QoE. That is, the network-induced contribution is the contribution, which may be in the responsibility of the ISP.
According to this implementation form, the network-induced contribution may be 100% (i.e., the determined QoE degradation is only caused by network issue(s)), or may be 0% (i.e., there is no contribution of the network to the QoE degradation). A distinction may be further made, whether the network-induced contribution is above 50% of the total contribution to the determined QoE degradation, or equal to or less than 50%. The network-induced contribution may, however, be the largest contribution, even if its contribution is less than 50% (e.g., among a contribution if network, content, or another cause).
In an implementation form of the first aspect, the device is further configured to: determine whether the QoE degradation corresponds to a change of the content metrics; and determine the contribution information, wherein the contribution information is indicative of the network-induced contribution being the largest contribution to the determined QoE degradation, if the QoE degradation does not correspond to a change of the content metrics.
In particular, the device may determine that the QoE degradation corresponds to a change of one or more content metrics, if the QoE degradation is concomitant with a change of the one or more content metrics, or if the QoE degradation occurs within a certain time period of the change of the one or more content metrics. Generally, a time relationship between the QoE degradation and the change of the one or more content metrics may be determined by the device, for instance, if one of the two shortly (within the certain time period) follows the other.
In an implementation form of the first aspect, the device is further configured to: determine whether the QoE degradation corresponds to a change of the content metrics; and determine the contribution information, wherein the contribution information is indicative of the network-induced contribution being not the largest contribution to the determined QoE degradation, if the QoE degradation corresponds to a change of the content metrics.
In an implementation form of the first aspect, the device is further configured to: obtain the content metrics by measuring the content of the web applications.
In particular, a change of the content of the one or more web applications (over time) may be obtained by the device. The device may perform active measurements to obtain the one or more content metrics, for instance, from one or more web pages of the web applications.
In an implementation form of the first aspect, the device is further configured to: obtain the user metrics by monitoring network traffic.
In particular, a change of the one or more user metrics (over time) may be obtained by the device, for instance, may be collected by the device. The device may perform passive measurements to obtain these user metrics, for instance, from encrypted or unencrypted web traffic.
In an implementation form of the first aspect, the device is further configured to, if the contribution information is indicative of the network-induced contribution being the largest contribution to the determined QoE degradation: obtain one or more network metrics, the network metrics being related to a performance of the network; and determine whether a change of at least one of the network metrics corresponds to the QoE degradation.
In particular, a change of the one or more network metrics (over time) may be obtained by the device, for instance, may be collected by the device. The device may perform passive and/or active measurements to obtain these network metrics, for instance, from other network elements.
In an implementation form of the first aspect, the device is further configured to: report the at least one of the network metrics corresponding to the QoE degradation; and/or rank the at least one of the network metrics corresponding to the QoE degradation according to a predefined criterion.
In an implementation form of the first aspect, the device is further configured to: report an unidentified network issue, if there is no change of at least one of the network metrics corresponding to the QoE degradation.
In an implementation form of the first aspect, the device is further configured to: estimate a value of the network-induced contribution to the QoE degradation; and/or estimate a value of a content-induced contribution to the QoE degradation, wherein the content-induced contribution is caused by a content change of the content of the web applications.
In an implementation form of the first aspect, the device is further configured to
A raw user metric may be a time series of measurements of a user metric over one user or over a pool of users of the one or more web applications. A raw content metric may be a time series of measurements of a content metric over one machine or over a pool of machines. One or more processing operations may be carried out on the respective raw metric to obtain the corresponding metric, for instance, processing operations comprising CPD and OD operations.
In an implementation form of the first aspect, the user metrics comprise at least one of: one or more QoS parameters related to the QoE of the users; one or more QoE indicators indicative of the QoE of the users; one or more key quality indicators related to the QoE of the users.
The QoS parameters may be related to one or more web applications.
In an implementation form of the first aspect, the user metrics comprise at least one of: a page loading time of one or more web pages related to the web applications; a speed index of one or more web pages related to the web applications; a byte index of one or more web pages related to the web applications.
In an implementation form of the first aspect, the content metrics comprise at least one of:
A second aspect of the disclosure provides a method for detecting a network-induced contribution to a QoE degradation for one or more web applications, the method comprising: obtaining one or more user metrics, the user metrics being indicative of a QoE of one or more users of the web applications; obtaining one or more content metrics, the content metrics being indicative of a content of the web applications; determining a QoE degradation of the QoE of the users based on the one or more user metrics; and determining a contribution information indicative of a network-induced contribution to the determined QoE degradation based on the content metrics.
In an implementation form of the second aspect, the contribution information is indicative of one of: the network-induced contribution is the largest contribution to the determined QoE degradation; the network-induced contribution is not the largest contribution to the determined QoE degradation; the network-induced contribution is the only contribution to the determined QoE degradation; the network-induced contribution to the determined QoE degradation is zero.
In an implementation form of the second aspect, the method further comprises: determining whether the QoE degradation corresponds to a change of the content metrics; and determining the contribution information, wherein the contribution information is indicative of the network-induced contribution being the largest contribution to the determined QoE degradation, if the QoE degradation does not correspond to a change of the content metrics.
In an implementation form of the second aspect, the method further comprises: determining whether the QoE degradation corresponds to a change of the content metrics; and determining the contribution information, wherein the contribution information is indicative of the network-induced contribution being not the largest contribution to the determined QoE degradation, if the QoE degradation corresponds to a change of the content metrics.
In an implementation form of the second aspect, the method further comprises: obtaining the content metrics by measuring the content of the web applications.
In an implementation form of the second aspect, the method further comprises: obtaining the user metrics by monitoring network traffic.
In an implementation form of the second aspect, the method further comprises, if the contribution information is indicative of the network-induced contribution being the largest contribution to the determined QoE degradation: obtaining one or more network metrics, the network metrics being related to a performance of the network; and determining whether a change of at least one of the network metrics corresponds to the QoE degradation.
In an implementation form of the second aspect, the method further comprises: reporting the at least one of the network metrics corresponding to the QoE degradation; and/or ranking the at least one of the network metrics corresponding to the QoE degradation according to a predefined criterion.
In an implementation form of the second aspect, the method further comprises: reporting an unidentified network issue, if there is no change of at least one of the network metrics corresponding to the QoE degradation.
In an implementation form of the second aspect, the method further comprises: estimating a value of the network-induced contribution to the QoE degradation; and/or estimating a value of a content-induced contribution to the QoE degradation, wherein the content-induced contribution is caused by a content change of the content of the web applications.
In an implementation form of the second aspect, the method further comprises: obtaining one or more raw user metrics and/or one or more raw content metrics; and obtaining the user metrics and/or the content metrics by performing a CPD algorithm and/or an OD algorithm on, respectively, the raw user metrics and/or the raw content metrics.
In an implementation form of the second aspect, the user metrics comprise at least one of: one or more QoS parameters related to the QoE of the users; one or more QoE indicators indicative of the QoE of the users; one or more key quality indicators related to the QoE of the users.
In an implementation form of the second aspect, the user metrics comprise at least one of: a page loading time of one or more web pages related to the web applications; a speed index of one or more web pages related to the web applications; a byte index of one or more web pages related to the web applications.
In an implementation form of the second aspect, the content metrics comprise at least one of: a number of objects of one or more web pages related to the web applications; a page weight of one or more web pages related to the web applications.
The method of the second aspect and its implementation forms provide the same advantages described above for the device of the first aspect and its respective implementation forms.
A third aspect of the disclosure provides a computer program comprising a program code for performing the method according to the second aspect or any of its implementation forms, when executed on a computer.
A fourth aspect of the present disclosure provides a non-transitory storage medium storing executable program code which, when executed by a processor, causes the method according to the second aspect or any of its implementation forms to be performed.
It has to be noted that all devices, elements, units and means described in the present application could be implemented in the software or hardware elements or any kind of combination thereof. All steps which are performed by the various entities described in the present application as well as the functionalities described to be performed by the various entities are intended to mean that the respective entity is adapted to or configured to perform the respective steps and functionalities. Even if, in the following description of specific embodiments, a specific functionality or step to be performed by external entities is not reflected in the description of a specific detailed element of that entity which performs that specific step or functionality, it should be clear for a skilled person that these methods and functionalities can be implemented in respective software or hardware elements, or any kind of combination thereof.
The above described aspects and implementation forms will be explained in the following description of specific embodiments in relation to the enclosed drawings, in which:
The device 100 is configured to obtain one or more user metrics 101, wherein the one or more user metrics 101 are indicative of a QoE of one or more users of the one or more web applications. The device 100 is not limited by how it obtains these user metrics 101. For instance, the device 100 may monitor network traffic (particularly web traffic related to the one or more web applications), in order to obtain the one or more user metrics 101. The network traffic may be encrypted or non-encrypted. The device 100 may collect the user metrics 101 from the network traffic.
Further, the device 100 is configured to obtain one or more content metrics 102, wherein the one or more content metrics 102 are indicative of a content of the one or more web applications. The device 100 is not limited by how it obtains these content metrics 102. For instance, the device 100 may measure the content of the one or more web applications, in order to obtain the one or more content metrics 102. For instance, the device 100 may measure a number of objects of one or more web pages related to the one or more web applications, and/or a page weight of one or more web pages related to the one or more web applications, in order to obtain the one or more content metrics 101. Thereby, the one or more content metrics 102 may comprise the number of objects and/or may comprise the page weight.
Further the device 100 is configured to determine a QoE degradation 103 of the QoE of the one or more users, based on the one or more user metrics 101. For instance, the device 100 may detect a change in the one or more user metrics 101, and may determine that the change is associated with, or indicates, a degradation of the QoE of the users associated with the one or more user metrics 101. The QoE degradation 103 of the one or more users may be temporally correlated with the change (e.g., a degradation) of the one or more user metrics 101.
The device 100 is then configured to determine a contribution information 104, which is indicative of a network-induced contribution to the determined QoE degradation 103. The device 100 is configured to determine the contribution information 104 based on the one or more content metrics 102. The contribution information 104 may be indicative of one of: the network-induced contribution is the largest contribution to the determined QoE degradation 103, e.g., it contributes by more than 50%; the network-induced contribution is not the largest contribution to the determined QoE degradation 103, e.g., it contributes by less than 50%; the network-induced contribution is the only contribution to the determined QoE degradation 103, i.e., it contributes completely by 100%; or the network-induced contribution to the determined QoE degradation 103 is zero, i.e., it contributes 0%.
The device 100 may comprise a processor or processing circuitry (not shown) configured to perform, conduct or initiate the various operations of the device 100 described herein. The processing circuitry may comprise hardware and/or the processing circuitry may be controlled by software. The hardware may comprise analog circuitry or digital circuitry, or both analog and digital circuitry. The digital circuitry may comprise components such as application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), digital signal processors (DSPs), or multi-purpose processors.
The device 100 may further comprise memory circuitry, which stores one or more instruction(s) that can be executed by the processor or by the processing circuitry, in particular under control of the software. For instance, the memory circuitry may comprise a non-transitory storage medium storing executable software code which, when executed by the processor or the processing circuitry, causes the various operations of the device 100 to be performed.
In one embodiment, the processing circuitry comprises one or more processors and a non-transitory memory connected to the one or more processors. The non-transitory memory may carry executable program code which, when executed by the one or more processors, causes the device 100 to perform, conduct or initiate the operations or methods described herein.
According to the above description, the device 100, according to embodiments of the invention, may identify a QoE degradation 103 of one or more users to be specifically caused by a network issue (i.e., identify that it is network-induced), either partly or completely. For instance, the device 100 may use active web measurements to disentangle apparent issues, which are due to changes in the content. A possible device 100 synoptic will be described in the following with respect to
The device 100 may obtain input as follows:
The device 100 obtains the one or more content metrics 102. For instance, the device wo may comprise one or more active content monitor (AM) components 202 (shown to be in a “content domain” above the horizontal dotted line), which may perform an automated and/or periodic monitoring of the content of the one or more web applications. For example, the device wo may monitor a target set 201 of web pages of interest (e.g., a set T of web pages among all web pages W of the one or more web applications; e.g., the set T may comprise the top N (e.g. N=1000) web pages that are frequently contacted).
The device 100 may, in particular, extract one or more raw content metrics 102r from the web pages of interest, wherein these web pages may yield a stable and steady stream of metric data.
The device 100 may perform a CPD and/or OD algorithm method 203 on the one or more raw content metrics 102r, which may be preliminary used to reduce the metric data that later needs to be exported and combined. The CPD and/or OD method 203 may transform the raw one or more content metrics 102r (e.g., the content series C in
The device 100 further obtains the one or more user metrics 101. For instance, the device 100 may comprise one or more passive network monitor (PM) components 205 (shown to be in a “network domain” below the dotted line”), which may be deployed in the network and may comprise measuring applications for measuring QoS and/or QoE and/or KQI indicators for web application sessions of the users. For example, the device 100 may receive the one or more user metrics 101 based on monitoring network traffic, particularly web traffic of the web applications, at one or more routers 204 of the network. The routers 204 may be associated with the one or more users of the one or more web applications.
The device 100 may thereby use, as the PM components 205, existing and deployed entities or systems, which are configured to extract network and application metrics (wherein content properties are not available at the network level).
The device 100 may, in particular, extract one or more raw user metrics coir related to the one or more users.
The device 100 may perform a CPD and/or OD method 203 on the one or more raw user metrics 101r, which can be used to reduce the metric data that later needs to be exported and combined. The CPD and/or OD method 203 may transform the one or more raw user metrics 101r (e.g., the quality series Q in
The device 100 may further process the input as follows:
The device 100 may be configured to collect, and process and/or combine the obtained raw user metrics 101r and raw content metrics 102r (e.g., the time series C and Q in
The device 100 may then be configured to perform a method 207 to process changes of the content metrics 102 and the user metrics 101, respectively, to filter determined QoE degradations 103. Thereby, the determined QoE degradations 103 are determined based on the one or more user metrics 101. In particular, the device 100 is configured to determine the contribution information 104, which indicates a network-induced contribution to each determined QoE degradation 103. Based thereon, the device may be further configured to filter the determined QoE degradations 103, for example, for a subset of QoE degradations 103 caused by the network only (e.g., for reducing false positive detection). Further, the device 100 may explain their relative importance as follows.
The device 100 may extract the most probable “cause” of any determined QoE degradation 103 among all possible contributions including {content, network, unknown}, and the device 100 may possibly rank and measure their relative proportions.
Given a “cause” between the possible contributions including {content, network, unknown} of a determined QoE degradation 103, the device 100 may provide further extraction of the actual component or indicator, which has induced the change on the one or more user metrics 101, in terms of the most probable cause, ranking, and/or relative proportion.
The method performed by the device 100 may extract the most probable “cause” of a QoE degradation 103 among the possible contributions including {content, network, unknown}. The method may comprise comparing one or more time series of user events with content events, as reflected in the one or more user metrics 101 and the one or more content metrics 102, respectively. An underlying idea of the method performed by the device 100 is explained with respect to
The device 100 may be configured to track QoE metrics (i.e., a user metric 101) of a user or a pool of users of a web application. The user metric 101 may, for example, comprise a PLT, and is represented by “U” in
The device 100 may determine a QoE degradation 103 based on the user metric 101. When a QoE degradation 103 is not observed, the device 100 may determine that there is no QoE issue, and may provide an according output 301. When the device 100 determines a QoE degradation 103 for this user or pool of users, the device 100 may continue by checking the content of the web application.
In particular, the device 100 may track a content metric 102, for example, the number of objects of a web page of the web application, represented by “C” in
The QoE degradation 103 may correspond to a specific instant given by an algorithm, typically a CPD algorithm, over a univariate or multivariate time series. The term “concomitant” may correspond to either occurring around the same time (with around defined by a certain margin of error in time the device 100 allows), or a more complex algorithm for detecting concomitance.
The embodiments of the invention presented in this disclosure provide several advantage, for instance:
Integration of user metrics 101 apart from network metrics for evaluating QoE degradation 103. In most conventional solutions, the QoE degradation is identified only under the perspective of network degradation, which is not sufficient for full evaluation user QoE degradation.
Integration of content metrics 102 for evaluating QoE degradation 103. In most conventional solutions, network and user-based metrics are exclusively used to evaluate QoE degradation even for web pages with changing content, which is insufficient to ascertain that user QoE degradation is tied to network degradation.
Focus on QoE degradation rooted by network sources, for which ISPs are interested, which reduces two types of false alarms: a. QoE degradation caused by content change; b. network degradation which does not impact QoE.
Reduction of false alarm compared to network only solutions, and compared to user only solutions.
The present invention has been described in conjunction with various embodiments as examples as well as implementations. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this disclosure and the independent claims. In the claims as well as in the description the word “comprising” does not exclude other elements or steps and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.
This application is a continuation of International Application No. PCT/EP2020/077766, filed on Oct. 5, 2020. The disclosure of which is hereby incorporated by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/EP2020/077766 | Oct 2020 | US |
| Child | 18295469 | US |
