Patent Application
20110202593
The present invention relates in general to the wireless telecommunications field and, in particular, to a performance management system and method for controlling which client terminal(s) are to provide a Quality of Experience, QoE, report and controlling a requested level of detail for the QoE report(s).
The following abbreviations are herewith defined, at least some of which are referred to within the following description of the prior art and the present invention.
Performance management is an important issue in wireless communication networks. Network operators should have feedback on the quality of the services they provide towards their users/subscribers. The network operators based on the feedback can take actions to assure that the agreed level of performance is reached. To fulfill this requirement, the network operators deploy a Performance Management (PM) system in the wireless communication network to control the performance monitoring, process the results, and act if performance degradation is detected.
The performance monitoring processes can be categorized into two main groups. The first group is known as network level monitoring where the performance data is collected in the network nodes/servers and sent to the PM system. Network level monitoring is performed via logging specific events in the nodes/servers or capturing and parsing data packets on aggregated interfaces. The PM system can process the measured raw data further to provide aggregated data (e.g. event counters, flow records, etc.). The advantage of network level monitoring is its flexibility and scalability, i.e. a few measurement points are sufficient to cover the whole area. The drawback of network level monitoring is that it is hard to obtain end-to-end (E2E) performance measurements for the individual client terminals.
The second group is known as terminal reporting where the performance data is collected by the client terminals themselves which make measurements related to the perceived quality and send the results to a server. The advantage of client terminal reporting is that it certainly provides E2E quality measurements. However, it is not feasible to request quality reports from all client terminals since this would impact the quality perceived by the users and also impact the network capacity and the server load.
In 3GPP networks the data available from network level monitoring for the PM system can be classified as follows (for example):
The above classification of PM data enables the PM system to localize problems/faults that occurred in the network and to perform root-cause analysis to find out the reasons of the problems/faults. For example, the performance degradation can be localized in certain cells or it might appear only for certain terminal types. Also, the performance degradation can be directly connected to poor radio conditions, inappropriate RBS/cell settings, congestion, bad terminal configuration, etc.
The PM system in addition to obtaining network level measurements should also be able to obtain information on performance from the terminal based measurements. There is a standard way of reporting terminal based measurements in the case of streaming media applications in 3GPP networks. In this case, an application server (e.g., PSS server, HTTP server, MBMS server) after negotiating the parameters of reporting with the client terminal will receive a Quality of Experience (QoE) report which is sent regularly from the client terminal. The following standardized services associated with streaming media have defined processes for the client terminals to send QoE reports to the application server:
In this document, the term application server and client terminal can mean either PSS application server and PSS client terminal or MBMS application server and MBMS client terminal.
In these two services, the main parameters such as rate, range, resolution of the QoE reports for different metrics are configured via an initial negotiation between the client terminal and the application server. According to prior art, the QoE reports are sent from the client terminal to the application server. The QoE reports on the application server are available for the PM system. The PM system fetches the QoE reports and uses them along with the existing network performance measurement results in fault localization and root-cause analysis. In the prior art, it is also up to the application server to decide which client terminals will send a QoE report and on what level. In particular, the application server controls and sets a percentage of the client terminals that will send the QoE reports and the level of details to provide in the QoE report.
Referring to
Referring to
Referring to
The PM system 118 needs the client terminal's QoE reports to help it perform the fault localization and root-cause analysis within the network 110. However, the PM system 118 does not have the control of the client terminal reporting measurements. The application server 106 has the control of the client terminal reporting measurements but it has no information on other network conditions to perform fault localization. This uncoordinated sampling significantly limits the possible use-cases of the PM system 118. For instance, since the application server 106 has to limit the amount of traffic caused by the client terminal's QoE reports by reducing the sampling ratio of the client terminals 104 that report QoE measurements. Plus, the PM system 118 for the same reason also limits the sampling ratio the high resolution node event reports that are requested from the RANs 102 and the CN 108. This means that there is no coordination between these samplings, thus the probability of properly matching QoE reports and network measurement reports is greatly reduced (i.e., if the sampling rates are 10%, the matching probability reduces to just 1%). Another problem with uncoordinated sampling is that the application server 106 will likely request to few client terminals 104 located in problematic areas, or terminal types etc. which otherwise should be providing QoE reports since the application server 106 is not focused on network management. Accordingly, there has been and is a need for enabling the PM system to control which client terminals are to provide QoE reports and also to control the level of detail in the QoE reports. This need and other needs have been satisfied by the present invention.
In one aspect of the present invention there is provided a method for performance management in a wireless communication network including at least a PM system, multiple client terminals, multiple network nodes, and an application server. The method is implemented by the PM system and includes the steps of: (1) selecting at least one client terminal to provide at least one QoE report; (2) determining a requested level of detail for the at least one QoE report; and (3) fetching the at least one QoE report from the application server after the at least one client terminal provides the at least one QoE report to the application server. An advantage of the method is that it enables the PM system to control which client terminals are to provide QoE reports and also control the level of detail in the QoE reports.
In another aspect of the present invention there is provided a PM system which includes: (1) a processor; and (2) a memory that stores processor-executable instructions where the processor interfaces with the memory and executes the processor-executable instructions to perform the following operations: (a) selecting at least one client terminal to provide at least one QoE report; (b) selecting a requested level of detail for the at least one QoE report which will be provided by the at least one client terminal; and (c) fetching the at least one QoE report from an application server after the at least one client terminal provides the at least one QoE report to the application server. Thus, the PM system controls which client terminals are to provide QoE reports and also controls the level of detail in the QoE reports.
In yet another aspect of the present invention there is provided a method for performance management in a wireless communication network including at least a PM system, multiple client terminals, multiple network nodes, and an application server. The method is implemented by the application server and includes the steps of (a) maintaining a media connected list of client terminals connected thereto; (b) maintaining a QoE settings list of the client terminals that have been requested to provide QoE reports together with a level of detail for the QoE reports; (c) maintaining a timer to control phasing out of the QoE setting list; (d) checking if the timer has expired; (i) if yes, emptying the QoE setting list; (ii) if no, checking if a connection changed event has occurred in which a connection was opened or closed to one of the client terminals; (iii) if yes, adding the one client terminal to the media connected list if the connection was opened or removing the one client terminal from the media connected list if the connection was closed and then sending an updated media connected list to the PM system; and (iv) if no connection changed event has occurred, checking if received a QoE update list from the performance management system and if yes then updating the QoE settings list according to the received QoE update list. This method enables the PM system to control which client terminals are to provide QoE reports and also control the level of detail in the QoE reports.
In yet another aspect of the present invention there is provided an application server which includes: (1) a processor; and (2) a memory that stores processor-executable instructions where the processor interfaces with the memory and executes the processor-executable instructions to perform the following operations: (a) maintaining a media connected list of client terminals connected thereto; (b) maintaining a QoE settings list of the client terminals that have been requested to provide QoE reports together with a level of detail for the QoE reports; (c) maintaining a timer to control phasing out of the QoE setting list; (d) checking if the timer has expired; (i) if yes, emptying the QoE setting list; (ii) if no, checking if a connection changed event has occurred in which a connection was opened or closed to one of the client terminals; (iii) if yes, adding the one client terminal to the media connected list if the connection was opened or removing the one client terminal from the media connected list if the connection was closed and then sending an updated media connected list to the PM system; and (iv) if no connection changed event has occurred, checking if received a QoE update list from the PM system and if yes then updating the QoE settings list according to the received QoE update list. This enables the PM system to control which client terminals are to provide QoE reports and also control the level of detail in the QoE reports.
In still yet another aspect of the present invention there is provided a method for performance management in a wireless communication network including at least a PM system, multiple client terminals, multiple network nodes, and an application server. The method is implemented by the application server and includes the steps of: (1) sending a message to the PM system, where the message contains a request for a QoE specification of a specific client terminal currently connected thereto; and (2) receiving a response from the PM system, where the response indicates that the specific client terminal is to provide a QoE report and a requested level of detail for the QoE report. This method enables the PM system to control which client terminals are to provide QoE reports and also control the level of detail in the QoE reports.
In yet another aspect of the present invention there is provided an application server which includes: (1) a processor; and (2) a memory that stores processor-executable instructions where the processor interfaces with the memory and executes the processor-executable instructions to perform the following operations: (i) sending a message to a PM system, where the message contains a request for a QoE specification of a specific client terminal currently connected thereto; and (ii) receiving a response from the PM system, where the response indicates that the specific client terminal is to provide a QoE report and a requested level of detail for the QoE report. This enables the PM system to control which client terminals are to provide QoE reports and also control the level of detail in the QoE reports.
In still yet another aspect of the present invention there is provided a method for performance management in a wireless communication network including at least a PM system, multiple client terminals, multiple network nodes, an application server, and an OMA-DM system. The method is implemented by the OMA-DM system and includes the steps of: (1) receiving a message from the PM system, where the message indicates at least one client terminal which is to provide at least one QoE report and the requested level of detail for the at least one QoE report; and (2) negotiating with the at least one client terminal to determine metrics of the requested level of detail for the at least one QoE report. This method enables the PM system to control which client terminals are to provide QoE reports and also control the level of detail in the QoE reports.
In still yet another aspect of the present invention there is provided an OMA-DM system which includes: (1) a processor; and (2) a memory that stores processor-executable instructions where the processor interfaces with the memory and executes the processor-executable instructions to perform the following operations: (i) receiving a message from a PM system, where the message indicates at least one client terminal which is to provide at least one QoE report and the requested level of detail for the at least one QoE report; and (ii) negotiating with the at least one client terminal to determine metrics of the requested level of detail for the at least one QoE report. This enables the PM system to control which client terminals are to provide QoE reports and also control the level of detail in the QoE reports.
Additional aspects of the invention will be set forth, in part, in the detailed description, figures and any claims which follow, and in part will be derived from the detailed description, or can be learned by practice of the invention. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as disclosed.
A more complete understanding of the present invention may be obtained by reference to the following detailed description when taken in conjunction with the accompanying drawings:
In the present invention, there is an enhanced PM system that controls which set of client terminals are to report (focused sampling) and the level of details of QoE reporting. The enhanced PM system can collect information on user performance from network level measurements and build-up a filtering rule set and based on this rule set and the network level measurements identify the client terminals which match the criteria and then requests the application server to have this set of client terminals provide QoE reports and the level of details of the QoE reports. The enhanced PM system does this by exchanging relevant data with the application server which collects the QoE reports. To achieve this, a communication scheme between the enhanced PM system and the application server is established (see
Since, the enhanced PM system can decide on the sampling focus, it will be capable of receiving client terminal's QoE reports from problematic areas, terminals, etc. For example, if the enhanced PM system is invoked to perform a detailed test focusing on a certain territory (e.g. particular cells, RBSs) then an algorithm is switched on that identifies all client terminals (via client IDs, e.g. IP addresses) initiating media streaming sessions from the investigated territory and then requests more detailed QoE reports from those client terminals. Similarly, the enhanced PM system can focus on certain terminal types or certain radio or traffic conditions. A detailed discussion about several different ways that the enhanced PM system can be configured and used to accomplish all of this has been provided below with respect to
Referring to
Referring to
In the discussions below, there are provided two different scenarios which are used to explain in detail how the PM system 402 has full control over the client terminal reporting measurements by communicating with the application server 408 (see
Referring to
1. The PM system 402 requests and receives relevant network measurement data from local stores of the RAN 404 or CN 410 (see communications 412 in
2. The client terminals 406 initiate new connections or terminate their existing connections to the application server 408.
3. The application server 408 provides an update on the IDs of those client terminals 406 connected to it (see connected message 414a in
4. The PM node 402 provides the application server 408 with the IDs of a sampled set of the connected client terminals 406 to request QoE reports from (see message 414b in
5. Negotiation of QoE metrics occurs between the selected client terminals 406 and the application server 408. These communications can be based on RTSP and SDP protocols.
6. The selected client terminals 406 send their QoE reports to the application server 408 periodically according to the QoE metric setup that was negotiated in message no. 5. This communication is based on RTSP or HTTP depending on the system 400 (PSS, MBMS).
7. The PM system 402 fetches the QoE reports from the application server 408 (see messages 414c and 414d in
The PM system 402 includes one or more processors 420 and at least one memory 422 (storage 422) that has processor-executable instructions where the processor(s) 420 are adapted to interface with the at least one memory 422 and execute the processor-executable instructions to receive messages 1 and 3 and process those messages to determine a sampling strategy and to provide a list of the sampled set of client terminals 406 and requested QoE metrics to the application server 408 in message 4 and then receive the sampled client terminal's QoE reports in message 7 from the application server 408 (note: the one or more processors 420 and the at least one memory 422 can be implemented, at least partially, as software, firmware, hardware, or hard-coded logic).
The applicant server 408 includes one or more processors 424 and at least one memory 426 (storage 426) that has processor-executable instructions where the processor(s) 424 are adapted to interface with the at least one memory 426 and execute the processor-executable instructions to receive message 2 which contains client terminal connection-disconnection information and send message 3 to the PM system 402 to update the list of the connected client terminals and then receive message 4 from the PM system 402 so know which client terminals 406 to negotiate the QoE metrics with during message 5 and then receive their QoE reports in message 6 and forward those QoE reports to the PM system 402 in message 7.
Referring to
1. The PM system 402 requests and receives relevant network measurement data from local stores of the RAN 404 or CN 410 (see communication 412 in
2. The client terminals 406 initiate new connections or terminate their existing connections to the application server 408.
3. The application server 408 provides an update on the IDs of those client terminals 406 connected to it (see list message 415a in
4. The PM node 402 provides the OMA-DM 502 with the IDs of a sampled set of the connected client terminals 406 to request QoE reports from (see control message 416a in
5. Negotiation of QoE metrics occurs between the selected client terminals 406 and the OMA-DM 502. These communications can be based on RTSP and SDP protocols.
6. The selected client terminals 406 send their QoE reports to the application server 408 periodically according to the QoE metric setup that was negotiated in message no. 5. This communication is based on RTSP or HTTP depending on the system 400 (PSS, MBMS or OMA-DM).
7. The PM system 402 fetches the QoE reports from the application server 408 (see messages 418a and 418b in
The PM system 402 includes one or more processors 420 and at least one memory 422 (storage 422) that includes processor-executable instructions where the processor(s) 420 are adapted to interface with the at least one memory 422 and execute the processor-executable instructions to receive messages 1 and 3 and process those messages to determine a sampling strategy and to provide a sampled set of client terminals 406 and requested QoE metrics to the OMA-DM 502 in message 4 and then receive the sampled client terminal's QoE reports in message 7 from the application server 408 (note: the one or more processors 420 and the at least one memory 422 can be implemented, at least partially, as software, firmware, hardware, or hard-coded logic).
The OMA-DM 502 includes one or more processors 428 and at least one memory 430 (storage 430) that has processor-executable instructions where the processor(s) 428 are adapted to interface with the at least one memory 430 and execute the processor-executable instructions to receive message 4 which indicates the set of client terminal 406 to provide QoE reports and the requested level of detail for the QoE reports from the PM system 402 and then in message 5 negotiates with the sampled client terminals 406 to determine to metrics for the QoE reports. This enables the PM system 402 to control which client terminals 406 are to provide QoE reports and also control the level of detail in the QoE reports.
In
A discussion about an exemplary scheme that can be used to update of the above lists (media_connected_list, qoe_settings_list) in both the PM system 402 and the application server 408 will be provided next with respect to
Referring to
Referring to
Referring to
Referring to
Referring to
1. The PM system 402 requests and receives relevant network measurement data from local stores of the RAN 404 or CN 410 (see comment 412 in
2. The client terminal 406 (client terminal X) initiates a new connection or terminates their existing connection to the application server 408.
3. The application server 408 sends a message to the PM system 402 requesting a QoE specification of client terminal X (see connected message 414a in
4. The PM node 402 sends the application server 408 a response indicating the specification of the QoE metrics for client terminal X after determining the preferred QoE metrics for client terminal X based on the conditions of client terminal X and the sampling algorithm (see control message 414b in
5. Negotiation of QoE metrics occurs between the application server 408 and client terminal X. In one case, the application server 408 can start the QoE negotiation after a timeout expires even if the response from the PM system 402 has not arrived so as not to affect the performance of client terminal X. These communications can be based on RTSP and SDP protocols.
6. Client terminal X sends their QoE report to the application server 408 periodically according to the QoE metric setup that was negotiated in message no. 5. This communication is based on RTSP or HTTP depending on the system 400 (PSS, MBMS).
7. The PM system 402 fetches the QoE report from the application server 408 (see messages 414c and 414d in
FIG. 12's arrows do not necessarily represent single messages but message sequences related to the subject of that specification communication. The white arrows represent those messages or message sequences that exist with the traditional PM system. The dark arrows represent those messages or message sequences that have been introduced in accordance with the present invention.
The PM system 402 includes one or more processors 420 and at least one memory 422 (storage 422) that has processor-executable instructions where the processor(s) 420 are adapted to interface with the at least one memory 422 and execute the processor-executable instructions to receive messages 1 and 3 and process those messages to determine a sampling strategy and to provide requested QoE metrics for client terminal X to the application server 408 in message 4 and then receive the sampled client terminal X's QoE report (and other sampled client terminal's QoE reports) in message 7 from the application server 408 (note: the one or more processors 420 and the at least one memory 422 can be implemented, at least partially, as software, firmware, hardware, or hard-coded logic).
The applicant server 408 includes one or more processors 424 and at least one memory 426 (storage 426) that has processor-executable instructions where the processor(s) 424 are adapted to interface with the at least one memory 426 and execute the processor-executable instructions to receive message 2 when client terminal X connects thereto and send the PM system 402 message 3 which contains a request for a QoE specification for client terminal X and then receive a reply in message 4 from the PM system 402 so can negotiate the QoE metrics in message 5 with client terminal X and then receive the QoE report in message 6 and forward the QoE report to the PM system 402 in message 7. This enables the PM system 402 to control which client terminals 406 are to provide QoE reports and also control the level of detail in the QoE reports.
Referring to
1. The PM system 402 requests and receives relevant network measurement data from local stores of the RAN 404 or CN 410 (see communications 412 in
2. The client terminals 406 initiate new connections or terminate their existing connection to the application server 408.
3. The application server 408 sends a message to the PM system 402 requesting a QoE specification of client terminal X (see control message 415a in
4. The PM node 402 sends the OMA-DM 502 a response indicating the specification of the QoE metrics for client terminal X after determining the preferred QoE metrics for client terminal X based on the conditions of client terminal X and the sampling algorithm (see control message 416a in
5. Negotiation of QoE metrics occurs between client terminal X and the OMA-DM 502. These communications can be based on RTSP and SDP protocols.
6. Client terminal X sends their QoE reports to the application server 408 periodically according to the QoE metric setup that was negotiated in message no. 5. This communication is based on RTSP or HTTP depending on the system 400 (PSS, MBMS or OMA-DM).
7. The PM system 402 fetches the QoE report from the application server 408 (see messages 418a and 418b in
FIG. 13's arrows do not necessarily represent single messages but message sequences related to the subject of that specification communication. The white arrows represent those messages or message sequences that exist with the traditional PM system. The dark arrows represent those messages or message sequences that have been introduced in accordance with the present invention.
The PM system 402 includes one or more processors 420 and at least one memory 422 (storage 422) that has processor-executable instructions where the processor(s) 420 are adapted to interface with the at least one memory 422 and execute the processor-executable instructions to receive messages 1 and 3 and process those messages to determine a sampling strategy and to provide client terminal X's requested QoE metrics to the OMA-DM 502 in message 4 and then receive the sampled client terminal X's QoE reports (and other sampled client terminal's QoE reports) in message 7 from the application server 408 (note: the one or more processors 420 and the at least one memory 422 can be implemented, at least partially, as software, firmware, hardware, or hard-coded logic).
The OMA-DM 502 includes one or more processors 428 and at least one memory 430 (storage 430) that has processor-executable instructions where the processor(s) 428 are adapted to interface with the at least one memory 430 and execute the processor-executable instructions to receive message 4 which indicates the client terminal X is to provide QoE reports and the requested level of detail for the QoE report from the PM system 402 and then in message 5 negotiates with client terminal X to determine metrics for the QoE reports. This enables the PM system 402 to control which client terminals 406 are to provide QoE reports and also control the level of detail in the QoE reports.
In view of the foregoing, one skilled in the art will appreciate that the PM system 402 enables network operators to make passive e2e tests focusing on a certain area. For example QoE reports can be requested by the PM system 402 from the client terminals 406 located in the same cell, from client terminals 406 of the same type, from client terminals 406 that have the same subscription type, etc. In one exemplary use case, client terminal X performs a Primary PDP Context Activation. This event is logged in the PM system 402 which contains all the data obtained from the signaling procedure which can come from the network (either interface measurements or node logs). Then, the same client terminal X initiates a connection towards the QoE enabled application server 408. The application server 408 sends a request to the PM system 402 for its QoE preference for client terminal X. Based on the QoE sampling algorithm (e.g. focusing on a certain set of cells), the PM system 402 makes a decision to request QoE reports from client terminal X and determines the level of details of the QoE reporting. The PM system 402 then sends a response to the application server 408 with the QoE specifications of client terminal X. In the QoE negotiation between client terminal X and the application server 408, the QoE report is requested from client terminal X based on the QoE preferences received from the PM system 402 (see
UC1: test e2e performance in upgraded cells
UC2: test newly introduced user equipments
UC3: test e2e performance of users of a given subscription
UC4: test e2e performance under poor radio conditions
Although several embodiments of the present invention have been illustrated in the accompanying Drawings and described in the foregoing Detailed Description, it should be understood that the invention is not limited to the disclosed embodiments, but instead is also capable of numerous rearrangements, modifications and substitutions without departing from the present invention that as has been set forth and defined within the following claims.
