Field
The present disclosure is generally directed to network performance, and more specifically, to systems and methods to understand the interplay and infer the status of the wireless network parameters from monitoring higher layer transport protocol parameters.
Related Art
In the related art, IEEE 802.11 technology is gaining increasing attention as the solution to provide ubiquitous connectivity to both indoors and outdoors situations on par with cellular networks. The success of this technology continues to grow as high speed versions are produced (e.g., 802.11ac, 802.11ad) and new market opportunities such as public wireless hotspots (e.g. cable wireless) are explored. The considerable increase in the number of users and the demand for high speed high bandwidth applications requires planning of the networks and design of the mechanisms to improve the quality of experience of the users. Maximization of the user quality of experience may require the development of an accurate model to analyze and subsequently pinpoint the pain points in the network.
Related art implementations have focused on analyzing or measuring the performance of the 802.11 network, or the Transmission Control Protocol (TCP) performance for 802.11 networks. To monitor the end user wireless quality, related art implementations involve injecting a scriplet into hypertext transmission protocols (HTTP) requests to periodically test the latency of HTTP requests from the mobile devices to the application server. An example of such a related art implementation can be found, for example, in U.S. Pat. No. 8,583,777, herein incorporated by reference in its entirety for all purposes. Related art implementations also involve a system where data gathering software is installed on the wireless device for collecting device parametric data, network parametric data and event data. An example of such a related art implementation can be found, for example, in U.S. Pat. No. 6,745,011, herein incorporated by reference in its entirety for all purposes.
In another related art implementation, the client downloads an active control measuring tool object in response to a request for content from the server to make network measurements, via direct socket access and returns the measurement results. An example of such a related art implementation can be found, for example, in U.S. Patent Publication No. 2011/0119370, herein incorporated by reference in its entirety for all purposes.
Aspects of the present disclosure include an apparatus, which can involve a memory configured to store a function for determining the quality of a wireless network from a network involving a wired network and a wireless network, the quality determined based on one or more packet flow key performance indicators of the network; and a processor, configured to obtain packet flows from the network; extract one or more packet flow key performance indicators from the packet flows; and determine quality of the wireless network from the function based on the extracted one or more packet flow key performance indicators.
Aspects of the present disclosure further include a method, which can involve managing a function for determining the quality of a wireless network from a network involving a wired network and a wireless network, the quality determined based on one or more packet flow key performance indicators of the network; obtaining packet flows from the network; extracting one or more packet flow key performance indicators from the packet flows; and determining quality of the wireless network from the function based on the extracted one or more packet flow key performance indicators.
Aspects of the present disclosure further include a non-transitory computer readable medium, storing instructions for executing a process which can involve managing a function for determining quality of a wireless network from a network involving a wired network and the wireless network, the quality determined based on one or more packet flow key performance indicators of the network; obtaining packet flows from the network; extracting one or more packet flow key performance indicators from the packet flows; and determining quality of the wireless network from the function based on the extracted one or more packet flow key performance indicators.
Aspects of the present disclosure further include an apparatus, which can involve a means for managing a function for determining quality of a wireless network from a network involving a wired network and the wireless network, the quality determined based on one or more packet flow key performance indicators of the network; means for obtaining packet flows from the network; means for extracting one or more packet flow key performance indicators from the packet flows; and means for determining quality of the wireless network from the function based on the extracted one or more packet flow key performance indicators.
The following detailed description provides further details of the figures and example implementations of the present application. Reference numerals and descriptions of redundant elements between figures are omitted for clarity. Terms used throughout the description are provided as examples and are not intended to be limiting. For example, the use of the term “automatic” may involve fully automatic or semi-automatic implementations involving user or administrator control over certain aspects of the implementation, depending on the desired implementation of one of ordinary skill in the art practicing implementations of the present application.
There is no related art implementation to model and infer the interplay between the transport layer parameters and the wireless parameters such as the signal level, the link quality or the interference level. Understanding these interactions can facilitate identifying and explaining the performance issues in the network. None of the related art implementations involve implementations for monitoring the wireless network performance from higher layer transport protocol metrics using a software agent at the network side.
The surge in the number of users, and the increase in multimedia streaming traffic on networks, leaves the network operators and planners straining for resources to meet the increasing demand in capacity and avoid congestion, both on the Ethernet Backhaul and the wireless access link. In an effort to guarantee an acceptable quality of experience for the users, example implementations are directed to facilitating network operators to gain insight into the quality of the wireless channels to the served users. The example implementations involve a system to infer the state of the wireless access link to the individual users from the access point (AP) by monitoring and analyzing the higher layer transport traffic end to end performance at the network side.
Example implementations involve a quality analysis and optimization module at the network side that monitors and analyzes the end to end performance of the network and infers the status of the wireless access link to the individual users from the AP.
Example implementations further involve network planning to improve the user Quality of Experience (QoE) based on the output of the analysis tool.
The analysis entity 203 receives the KPIs from the KPI and feature extraction entity 202 and derives the quality of the wireless channels to the individual or groups of user equipment (UEs). The analysis is based on a developed model for advanced analytics that takes into account the interaction and correlation between the extracted KPIs and the KPIs for the wireless channel. The analysis entity 203 also analyzes the wireless channel KPIs and the extracted KPIs to perform for example data traffic classification and analysis, as well as insights into the health of the network from an end to end perspective. The data traffic classification may include for example classification of the users according to their traffic or link quality. Another example of data analytics performed is to classify the users according to their expected quality of experience depending on the application, e.g. which users can expect good quality of experience when performing multimedia streaming, which users can expect good quality of experience when performing wireless calling, and so on.
The optimization entity 204 takes input from the analysis entity 203 and provides recommendations for network planning and optimization based on the analytics performed on the user and network data. Further details of the optimization conducted in the optimization entity 204 are provided with respect to
The quality analysis and optimization module 200 shown in
For the analysis entity 203 in
In
The training module then decides based on the calculated network KPIs if the measured packet captures are conclusive to derive the quality of the end-to-end performance of the session or more measurements are needed at 404. The measurements can be judged as conclusive, for example, if there are enough data points collected to be able to make a decision about data traffic modeling. Such a decision can be made if an accurate description of the statistical characteristics of the traffic on the network can be reached, for example, if the collected data points from the packet flows can be used to capture the accurate characteristics of the network. If not enough data points can be used to capture the characteristics of the network to a desired accuracy level, then this can lead to an underestimation or over estimation of the network characteristics, in terms of utilization, congestion level, connection quality, and so on. Example decision can be made whenever the performance metrics or extracted KPIs in 403 for each user do not show variations in performance. When data is measured in an office space for example, the variations in the performance measured in 403 can be due to varying environment such as people moving or moving obstacles, this affects the performance of the transmission, such that the throughput or latency or TCP anomalies. If only one measurement is made corresponding to one short transmission session is made at any given location, this may not give a good average indication of the performance of the network at a particular location. More measurements are needed at that point such that the aggregate performance gives a relatively steady state indication of the performance of the network at that particular location. The number of measurements needed for example (ex. 10 measurements) is judged by how varying the results are from one measurement to the other. For example 10 measurements can give conclusive results (404) such that the computed KPIs in 403 are comparable from one measurement to the next.
In an example of a conclusive determination, a confidence level threshold can be applied to the sample set, which can require a sufficient number of measurements for the confidence level to be met. In such an example implementation, the confidence level threshold is set according to the desired implementation, and at 404, the confidence level is calculated for all of the measurements received. If the confidence level is met, then the results can be deemed conclusive, otherwise, the results can be deemed inconclusive. Other implementations (e.g. threshold level of measurements, etc.) can also be implemented depending on the desired implementation.
If the network KPIs extracted from packet captures are deemed conclusive (Yes), the training module collects the corresponding information obtained from the software agent at the user (402) and the derived KPIs (403). The exercise of collecting the information can be repeated with more measurements, at different times of the day, for different types of packet flows, and a different network load (number of users in the network). The collected information is used to derive a model for the wireless link characteristics as a function of the KPIs in 403. An example model links the quality of the link at user k with the KPIs in 403.
Q(uk)=f(RTTk,Thk,L, etc.) (1)
Where uk is user k, Q is the quality of link at user k, RTTk is the round trip time of the session at user k, Thk is throughput at user k, L is the network load at the access point that user k is connected to.
The number of measurements collected and the number of locations and types of packet flows determine the quality of the derived training model. Collected information per user or per location can be repeated to capture a steady state behavior of the network, similar to 404. The number of data points collected also affects the quality of the training data used to derive the model used for the quality analysis and optimization module 200. The validity of the data points used to derive the model depends on the number of measurements, the number of sessions used for each measurement location, the type of users, the network load, and so on. The number and quality of the data points collected can be judged conclusive or not (406) if the number and quality are enough to derive a model for more users in the network using the training data.
Similar to 404, the conclusiveness of the number and quality of the data points of 406 can be determined based on the application of a confidence level threshold can be applied to the data points, which can require a sufficient number of data points for the confidence level to be met. In such an example implementation, the confidence level threshold is set according to the desired implementation, and at 406, the confidence level is calculated for all of the data points received. If the confidence level is met, then the results can be deemed conclusive, otherwise, the results can be deemed inconclusive. Other implementations (e.g. threshold level of data points, etc.) can also be implemented depending on the desired implementation.
The derivation of the model for the wireless link quality as a function of the KPIs in 403, can be done using mathematical modeling taking into account the relation between the end to end performance captured in the KPIs and the wireless link quality, it can also be done using statistical analysis that fits the collected data in 405 in a given model and predicts the quality of the users links as a function of the end to end KPIs in 403. Such a model can be for example a regression model whose parameters are chosen to fit the collected KPIs in 403 as a function of the parameters in 402.
An example derivation of a model of the wireless link statistics SINR as a function of the end to end round trip time of the transmission session for a given user at a given location using mathematical modeling is as follows. The round trip time is calculated in step 403 from the network software agent. The SINR of users is obtained in 402 from the software agent at the user. For a given user with a given SINR, in step 403 a set of network KPIs is obtained. A model can be derived for RTT as a function of SINR to be later used in the quality analysis and optimization module 200. The RTT is a function of the probability of packet loss in the transmission as well as the number of retransmission attempts, the amount of fragmentation at the Multiple Access Layer (MAC). Assuming a single user transmission, in the absence of probability of packet loss due to collision, the probability of packet loss is a function of the probability of error. The probability of error is related to the bit error rate. The bit error rate, depending on the channel model used (Rayleigh Channel, Pathloss channel model), is a function of the SINR at the user This links the RTT of a user at a particular location, subject to a particular channel to the SINR of that user.
The resulting model in the quality analysis training module expresses the wireless link quality as a function of the end to end network KPIs. This derived model is used by the administrators in the quality analysis and optimization module 200.
To implement the derived model for the quality of the wireless link, the quality analysis and optimization module 200 can make use of a look up table in the analysis entity 203 to map the quality of the wireless link as a function of the measured higher layer KPIs. This look-up table is a simple implementation of the statistical analysis obtained using training. The quality analysis and optimization module 200 can further implement a mathematical formula in the analysis entitiy 203 as in equation 1 for example to derive the quality of the wireless link as a function of the end-to-end KPIs.
The model derived from the training exercise either by passive or active content is then used in the quality analysis and optimization module to infer and optimize the quality of the wireless network.
At 602, the quality analysis and optimization module 200 derives statistics such as aggregate RTT, throughput, packet loss rate for individual user sessions from the statistics collected in 601. It extracts and computes KPIs from the higher layer transport protocol traces (ex. TCP and or UDP) used to transport the traffic to and from the users (wireless clients) Example KPIs are RTT, throughput, packet loss rate, latency, congestion window size, etc. using the measured packet captures from 601. Example calculations of KPIs include but not limited to the following given examples. The aggregate RTT for one packet flow session is computed for example by averaging over the RTT from each session between a server and a client. The server and the client can be users in the network or the server can be located on the network side or at the access point. The throughput of the download or upload session for one user for example, is measured by calculating the total amount of bytes transferred and the total delay of the transfer. The number of lost packets for a UDP-type traffic for example indicates the amount of packets sent but not received at the user.
The module 200 then derives, at 603, the KPIs of the wireless links from the computed network KPIs based on, for example, the derived network-wireless model derived in the quality analysis training module. The module 200 can make use of a look up table at 603 to map the computed KPIs in 602 to the quality of the wireless link as derived in the training module in
Table 1 can be created based on the derived model from
The received signal strength information (RSSI) is an indication of the power level at the receiver antenna. The higher the RSSI number, the better the signal, and the better the wireless link between the access point and the user. In the presence of multiple clients per AP, however, the RSSI is not an indication of the expected probability of error or the expected round trip time, as the probability of collision and the collision between different clients downloads can be another reason for degradation of the wireless channel performance. The signal to interference noise ratio, and the packet loss rate serve as other indications of the health of the wireless signal. Table 2 shows a lookup table that extracts the packet loss rate from the throughput, latency, and network load computed in 602. The numbers are given for illustration purposes.
The quality analysis and optimization module 200 can further implement a mathematical formula in the analysis entity 603 as in equation 1 for example to derive the quality of the wireless link as a function of the end-to-end KPIs. The derived wireless link performance model is then used for network optimization decisions as shown in
At 703, the result of the grouping and clustering exercise is used to check on the quality of the link for different users at different locations in the network corresponding to different APs. A check is performed to determine if the users for a certain AP at a certain location meet their QoE requirements. If so (Yes) then the flow ends, meaning no optimization needed, and the network is operating in a healthy non congested manner otherwise (No) the flow proceeds to 704 to re-visit network planning decisions according to the analysis performed in 702 and 703.
Below are examples for network analysis and optimization performed in 704 for module 204 based on the performed measurements and data analysis in 703.
In a first example for the case of mobile carrier wireless, where the cellular carrier is interested in offloading his traffic to a trusted wireless network deployed by the carrier itself. The users may monitor the strength of the wireless link or the air interface signal strength, and switch to wireless from the cellular network whenever the wireless signal is higher than a certain threshold. The users however may not get a good QoE while on the wireless network, resulting in unsatisfied customers. Using the quality analysis and optimization module 203 in accordance with example implementations, the expected quality of the air interface can be derived as a function of the end to end performance. The optimization module 204 can then provide recommendations of the type of applications that can be offloaded for a certain user to the wireless network, and what wireless air interface or channel width or number of streams the user can handle.
In another example, the quality of the wireless link can be considered good, but users continue to experience bad end to end performance, due for example to congestion at the backhaul link. In such an example, the module 204 can give recommendations for policy and network planning at the wired side as well the wireless side. Congestion can also be at the wireless link, when a large number of users are accessing the network, which leads to worsening of the end-to-end performance in network KPIs, worse packet loss rate, and probability of error at the wireless channel. The module 204 can be used to pinpoint the congestion problem and network planning decisions such as adding APs or changing traffic prioritizations can be done to relief the congestion problem.
In another example, the quality analysis and optimization tool module can be used to monitor the quality of the wireless channels, and pinpoint coverage holes. Coverage holes can be pinpointed for example when the wireless channel quality is consistently bad for a stationary user or a group of users throughout a long period of measurement and monitoring. This can be obtained without the need for obtaining the user location, in case that information is not available at the quality analysis and optimization module. For the case when the user location information is available at the network side, heatmaps can be generated from the coverage at different APs, this leads to network planning decisions related to deploying one or more APs at coverage holes locations to serve areas with no coverage. This optimization example can be applied for example to the case of cable wireless, where cable operators make use of residential access points to provide public access to subscribed users. It can also be applied to enterprise wireless environments, or other venue wireless, such as stadium wireless.
In another example, the quality analysis and optimization tool can further be used for energy efficiency optimization. The quality and usage of the wireless channels indicated by the tool can be used by the network planners to switch off some of the APs to save energy whenever they are not needed. This also reduces the interference that the APs might cause.
Computer device 805 can be communicatively coupled to input/user interface 835 and output device/interface 840. Either one or both of input/user interface 835 and output device/interface 840 can be a wired or wireless interface and can be detachable. Input/user interface 835 may include any device, component, sensor, or interface, physical or virtual, that can be used to provide input (e.g., buttons, touch-screen interface, keyboard, a pointing/cursor control, microphone, camera, braille, motion sensor, optical reader, and/or the like). Output device/interface 840 may include a display, television, monitor, printer, speaker, braille, or the like. In some example implementations, input/user interface 835 and output device/interface 840 can be embedded with or physically coupled to the computer device 805. In other example implementations, other computer devices may function as or provide the functions of input/user interface 835 and output device/interface 840 for a computer device 805.
Examples of computer device 805 may include, but are not limited to, highly mobile devices (e.g., smartphones, devices in vehicles and other machines, devices carried by humans and animals, and the like), mobile devices (e.g., tablets, notebooks, laptops, personal computers, portable televisions, radios, and the like), and devices not designed for mobility (e.g., desktop computers, other computers, information kiosks, televisions with one or more processors embedded therein and/or coupled thereto, radios, and the like).
Computer device 805 can be communicatively coupled (e.g., via I/O interface 825) to external storage 845 and network 850 for communicating with any number of networked components, devices, and systems, including one or more computer devices of the same or different configuration. Computer device 805 or any connected computer device can be functioning as, providing services of, or referred to as a server, client, thin server, general machine, special-purpose machine, or another label.
I/O interface 825 can include, but is not limited to, wired and/or wireless interfaces using any communication or I/O protocols or standards (e.g., Ethernet, 802.11x, Universal System Bus, WiMax, modem, a cellular network protocol, and the like) for communicating information to and/or from at least all the connected components, devices, and network in computing environment 800. Network 850 can be any network or combination of networks (e.g., the Internet, local area network, wide area network, a telephonic network, a cellular network, satellite network, and the like).
Computer device 805 can use and/or communicate using computer-usable or computer-readable media, including transitory media and non-transitory media. Transitory media include transmission media (e.g., metal cables, fiber optics), signals, carrier waves, and the like. Non-transitory media include magnetic media (e.g., disks and tapes), optical media (e.g., CD ROM, digital video disks, Blu-ray disks), solid state media (e.g., RAM, ROM, flash memory, solid-state storage), and other non-volatile storage or memory.
Computer device 805 can be used to implement techniques, methods, applications, processes, or computer-executable instructions in some example computing environments. Computer-executable instructions can be retrieved from transitory media, and stored on and retrieved from non-transitory media. The executable instructions can originate from one or more of any programming, scripting, and machine languages (e.g., C, C++, C#, Java, Visual Basic, Python, Perl, JavaScript, and others).
Computer device 805 can be configured to implement the architecture as illustrated in
Processor(s) 810 can be configured to obtain packet flows from the network, extract one or more packet flow KPIs from the packet flows, and determine quality of the wireless network from the function based on the extracted one or more packet flow KPIs, as illustrated in
Some portions of the detailed description are presented in terms of algorithms and symbolic representations of operations within a computer. These algorithmic descriptions and symbolic representations are the means used by those skilled in the data processing arts to convey the essence of their innovations to others skilled in the art. An algorithm is a series of defined steps leading to a desired end state or result. In example implementations, the steps carried out require physical manipulations of tangible quantities for achieving a tangible result.
Unless specifically stated otherwise, as apparent from the discussion, it is appreciated that throughout the description, discussions utilizing terms such as “processing,” “computing,” “calculating,” “determining,” “displaying,” or the like, can include the actions and processes of a computer system or other information processing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other information storage, transmission or display devices.
Example implementations may also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may include one or more general-purpose computers selectively activated or reconfigured by one or more computer programs. Such computer programs may be stored in a computer readable medium, such as a computer-readable storage medium or a computer-readable signal medium. A computer-readable storage medium may involve tangible mediums such as, but not limited to optical disks, magnetic disks, read-only memories, random access memories, solid state devices and drives, or any other types of tangible or non-transitory media suitable for storing electronic information. A computer readable signal medium may include mediums such as carrier waves. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Computer programs can involve pure software implementations that involve instructions that perform the operations of the desired implementation.
Various general-purpose systems may be used with programs and modules in accordance with the examples herein, or it may prove convenient to construct a more specialized apparatus to perform desired method steps. In addition, the example implementations are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the example implementations as described herein. The instructions of the programming language(s) may be executed by one or more processing devices, e.g., central processing units (CPUs), processors, or controllers.
As is known in the art, the operations described above can be performed by hardware, software, or some combination of software and hardware. Various aspects of the example implementations may be implemented using circuits and logic devices (hardware), while other aspects may be implemented using instructions stored on a machine-readable medium (software), which if executed by a processor, would cause the processor to perform a method to carry out implementations of the present application. Further, some example implementations of the present application may be performed solely in hardware, whereas other example implementations may be performed solely in software. Moreover, the various functions described can be performed in a single unit, or can be spread across a number of components in any number of ways. When performed by software, the methods may be executed by a processor, such as a general purpose computer, based on instructions stored on a computer-readable medium. If desired, the instructions can be stored on the medium in a compressed and/or encrypted format.
Moreover, other implementations of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the teachings of the present application. Various aspects and/or components of the described example implementations may be used singly or in any combination. It is intended that the specification and example implementations be considered as examples only, with the true scope and spirit of the present application being indicated by the following claims.