Online meetings with applications such as TEAMS®, ZOOM®, etc., play an ever-increasing role in our daily work and personal lives. These applications for example allow remote employees to work and collaborate closely through video or voice conference calling with meetings, technical sharing, status reviews, etc.
Aspects of this disclosure include a system and method for managing quality issues experienced during online meetings.
A first aspect of the disclosure provides a system having a memory and a processor coupled to the memory and configured to manage quality issues for a set of clients participating in an online meeting. A process includes receiving a report from a first client of a quality issue associated with a second client. Once reported, the process includes obtaining and evaluating performance data from the first client to determine whether the first client is responsible for the quality issue. In response to a determination that the first client is not responsible for the quality issue, requesting an indication from a set of other clients in the online meeting whether the other clients are experiencing the quality issue with the second client. In response to a determination that the second client is responsible for the quality issue, notifying the second client of the quality issue.
A second aspect of the disclosure provides a method of managing quality issues for a set of clients participating in an online meeting. The method includes: receiving a report from a first client of a quality issue associated with a second client; obtaining and evaluating performance data from the first client to determine whether the first client is responsible for the quality issue; in response to a determination that the first client is not responsible for the quality issue, requesting an indication from a set of other clients in the online meeting whether the other clients are experiencing the quality issue with the second client; and in response to a determination that the second client is responsible for the quality issue, notifying the second client of the quality issue.
The illustrative aspects of the present disclosure are designed to solve the problems herein described and/or other problems not discussed.
These and other features of this disclosure will be more readily understood from the following detailed description of the various aspects of the disclosure taken in conjunction with the accompanying drawings that depict various embodiments of the disclosure, in which:
The drawings are intended to depict only typical aspects of the disclosure, and therefore should not be considered as limiting the scope of the disclosure.
Embodiments of the disclosure provide technical solutions for managing quality issues experienced by users during online meetings. Online meeting platforms such as TEAMS and ZOOM generally operate in a client-server model in which a server manages a session amongst a set of clients, and users interact with respective clients (i.e., applications running on client devices). In a typical scenario, one user will act as the host or presenter and invite other users to participate in an online meeting. Once the online meeting is active, users can share audio and/or video with other users, share screen content, chat, etc.
However, because clients connect to the server under different circumstances (e.g., different hardware, different communication bandwidths, different locations, etc.) it is not unusual for one or more users to experience quality issues when participating in an online meeting. For example, a smartphone user with limited cell service and Wi-Fi may be more likely to experience technical issues than an office desktop user with an ethernet connection. Quality issues that users might experience include, e.g., bad voice quality of the speaker, lack of video, video freezing, an unstable connection, etc.
Because of the nature of online meetings, addressing quality issues during the meeting can be a challenge. For example, if a user is experiencing an issue, e.g., cannot hear the speaker clearly, the user could interrupt the other participants to determine the cause. In some cases, the user may disrupt the flow of the meeting only to learn that no one else is experiencing the issue. Accordingly, rather than interrupting the flow of the meeting, the user may elect to not interrupt the meeting and miss important content, even though others may also be experiencing the same issue. In other cases, a user may be hosting the meeting or presenting, and not be aware for some time that others cannot hear clearly, thus wasting time for everyone.
The present approach provides an interactive and dynamic technical solution for managing quality issues experienced by users during an online meeting. In various embodiments, the participants can report a quality issue to the server during a meeting via a user interface, e.g., by clicking a button. Upon receiving the report, the server acts immediately to help determine if the cause of the issue is on the reporting user's end. The server can also trigger a voting request via the interface tool to other participants to facilitate a comprehensive judgement as to the cause of the issue. Once a likely cause of the issue is determined, alerts and countermeasures are provided to the impacted participants such that appropriate actions can be taken. Additionally, the server can record the event details in a database, e.g., with a user's profile, including type of quality issue reported and countermeasures taken, thus allowing the user to be reminded of the issue in the future when using the same network or device.
Online meeting platform 32, which manages meeting sessions, includes issue management features such as: (1) a performance data collection system 34 that collects performance information from clients 14, 14′; (2) an issue management system 36 that manages issues reported by clients 14, 14′, issues alerts, and provides issue resolution suggestions; (3) a performance data analysis system 38 that analyzes performance data when issues are reported to determine a cause; and (4) an event database that stores issue based events, including issue type, countermeasures taken, resolutions, etc.
Upon receiving the issue report from Frank, the performance data collection system 34 on server 30 triggers a current (e.g., real-time) query to Frank's performance data reporting system 22 to retrieve performance data such as network quality data, network round trip time, bandwidth, jitter, packet loss, client workload (e.g., CPU and memory usage), etc. Once the query results are obtained by the server 30, performance data analysis system 38 analyzes the results together with accumulated benchmark data of Frank to make a quick judgement as to potential causes. If the problem appears to be with Frank, an alert will be sent to Frank's client by issue management system 36 to indicate the problem is at Frank's end (i.e., with Frank's client, client device, network connection, etc.). As shown in
If, however the issue does not appear to be on Frank's end, issue management system 36 will trigger a voting request to all of the other participants (except Alice) to see if the others are experiencing similar issues with Alice. For instance, as shown in
In a scenario where a quality issue is reported by a reporting user, but the issue is not with the reporting user, any type of vote gathering process and voting algorithm may be implemented to judge whether an issue exists with the reported user (e.g., Alice). In some embodiments, the other participates can send back indications in which some vote “good” and others vote “bad”. The following table provide an example of voting algorithm implemented by issue management system 36 for determining if the vote is a success (i.e., there appears to be an issue with the reported user).
Referring now to
At S5, performance data analysis system 38 determines whether the issue is at the reporting user's end (i.e., Frank's end). In one embodiment data analysis system 38 compares/analyzes the current performance data with benchmark data to determine if the problem appears to be with the reporting user. For example, if the network round trip time appears to be slowing down or falls below a threshold, or if packet loss is detected, or if memory usage is significantly above the benchmark data values, then the problem can be judged to be with the reporting user. If yes at S5, an alert and associated countermeasures are sent to the reporting user at S6. After a brief period of time, a check is made at S7 to see if the issue is resolved, i.e., whether an implemented countermeasure worked. This determination may be done with a query to the reporting user. If the issue is resolved, the event ends at S8 with event details optionally being saved in the event database 40 with the user profile. The event details can be provided to the user in the future to head off similar potential issues.
If the issue does not appear to be with the reporting user at S5, or the issue is not resolved at S7, i.e., the countermeasures did not work, a voting process is initiated at S9 to ascertain whether the other users are experience a similar quality issue. If the voting succeeds at S10 based on a voting algorithm, i.e., enough other users are having the same issue (e.g., with Alice), current performance data is obtained from the reported user (i.e., Alice) at S11. At S12, the current performance data is analyzed (e.g., in view of previously collected benchmark data for the reported user) to determine if there is an issue with the reported user. If yes at S12, an alert and countermeasures are sent to the reported user at S13. After a brief period of time, a determination is made at S14 (e.g., by analyzing new performance data or sending queries to one or more users) as to whether the issue is resolved. If resolved at S14, then event ends and the event details are optionally saved in the event database for future use.
In the case where the voting does not succeed at S10, the issue is not with the reported user at S12, or the issue is not resolved at S14, the issue can be classified an unresolved event at S15. In this case, some additional actions can be taken, e.g., notifying all the users, notifying an administrator, making recommendations such restarting the meeting for the impacted user(s), etc. It is also understood the scenarios described herein are not intended to be limiting. For instance, there may be situations where multiple users report a quality issue regarding a presenting user at the same time. In that case, the issue management system 36 can simply jump to S11 and obtain current performance data from the presenter and proceed accordingly.
Aspects of the approaches detailed herein accordingly provide an interactive mode, integrated into online meeting system, that allows for real-time user feedback of perceived quality issues to improve user experience. Comprehensive judgements can be made at the server 30, which can include the subject observations of the participants via a voting mechanism and the objective evaluation of performance data (e.g., network status and workload on client side). With this approach, the solution offers just-in-time and accurate evaluations, which can also be combined with other existing in-band quality detection mechanisms. Further, the described solutions result in minimal impact to ongoing meetings when quality issues arise.
It is understood that online meeting system can be implemented in any manner, e.g., as a stand-alone system, a distributed system, within a network environment, etc. Referring to
In some embodiments, the client machines 102A-102N communicate with the remote machines 106A-106N via an intermediary appliance 108. The illustrated appliance 108 is positioned between the networks 104, 104′ and may also be referred to as a network interface or gateway. In some embodiments, the appliance 108 may operate as an application delivery controller (ADC) to provide clients with access to business applications and other data deployed in a datacenter, the cloud, or delivered as Software as a Service (SaaS) across a range of client devices, and/or provide other functionality such as load balancing, etc. In some embodiments, multiple appliances 108 may be used, and the appliance(s) 108 may be deployed as part of the network 104 and/or 104′.
The client machines 102A-102N may be generally referred to as client machines 102, local machines 102, clients 102, client nodes 102, client computers 102, client devices 102, computing devices 102, endpoints 102, or endpoint nodes 102. The remote machines 106A-106N may be generally referred to as servers 106 or a server farm 106. In some embodiments, a client device 102 may have the capacity to function as both a client node seeking access to resources provided by a server 106 and as a server 106 providing access to hosted resources for other client devices 102A-102N. The networks 104, 104′ may be generally referred to as a network 104. The networks 104 may be configured in any combination of wired and wireless networks.
A server 106 may be any server type such as, for example: a file server; an application server; a web server; a proxy server; an appliance; a network appliance; a gateway; an application gateway; a gateway server; a virtualization server; a deployment server; a Secure Sockets Layer Virtual Private Network (SSL VPN) server; a firewall; a web server; a server executing an active directory; a cloud server; or a server executing an application acceleration program that provides firewall functionality, application functionality, or load balancing functionality.
A server 106 may execute, operate or otherwise provide an application that may be any one of the following: software; a program; executable instructions; a virtual machine; a hypervisor; a web browser; a web-based client; a client-server application; a thin-client computing client; an ActiveX control; a Java applet; software related to voice over internet protocol (VoIP) communications like a soft IP telephone; an application for streaming video and/or audio; an application for facilitating real-time-data communications; a HTTP client; a FTP client; an Oscar client; a Telnet client; or any other set of executable instructions.
In some embodiments, a server 106 may execute a remote presentation services program or other program that uses a thin-client or a remote-display protocol to capture display output generated by an application executing on a server 106 and transmit the application display output to a client device 102.
In yet other embodiments, a server 106 may execute a virtual machine providing, to a user of a client device 102, access to a computing environment. The client device 102 may be a virtual machine. The virtual machine may be managed by, for example, a hypervisor, a virtual machine manager (VMM), or any other hardware virtualization technique within the server 106.
In some embodiments, the network 104 may be: a local-area network (LAN); a metropolitan area network (MAN); a wide area network (WAN); a primary public network 104; and a primary private network 104. Additional embodiments may include a network 104 of mobile telephone networks that use various protocols to communicate among mobile devices. For short range communications within a wireless local-area network (WLAN), the protocols may include 802.11, Bluetooth, and Near Field Communication (NFC).
Elements of the described solution may be embodied in a computing system, such as that shown in
Processor(s) 302 may be implemented by one or more programmable processors executing one or more computer programs to perform the functions of the system. As used herein, the term “processor” describes an electronic circuit that performs a function, an operation, or a sequence of operations. The function, operation, or sequence of operations may be hard coded into the electronic circuit or soft coded by way of instructions held in a memory device. A “processor” may perform the function, operation, or sequence of operations using digital values or using analog signals. In some embodiments, the “processor” can be embodied in one or more application specific integrated circuits (ASICs), microprocessors, digital signal processors, microcontrollers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), multi-core processors, or general-purpose computers with associated memory. The “processor” may be analog, digital or mixed-signal. In some embodiments, the “processor” may be one or more physical processors or one or more “virtual” (e.g., remotely located or “cloud”) processors.
Communications interfaces 306 may include one or more interfaces to enable computer 300 to access a computer network such as a LAN, a WAN, or the Internet through a variety of wired and/or wireless or cellular connections.
In described embodiments, a first computing device 300 may execute an application on behalf of a user of a client computing device (e.g., a client), may execute a virtual machine, which provides an execution session within which applications execute on behalf of a user or a client computing device (e.g., a client), such as a hosted desktop session, may execute a terminal services session to provide a hosted desktop environment, or may provide access to a computing environment including one or more of: one or more applications, one or more desktop applications, and one or more desktop sessions in which one or more applications may execute.
As will be appreciated by one of skill in the art upon reading the following disclosure, various aspects described herein may be embodied as a system, a device, a method or a computer program product (e.g., a non-transitory computer-readable medium having computer executable instruction for performing the noted operations or steps). Accordingly, those aspects may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, such aspects may take the form of a computer program product stored by one or more computer-readable storage media having computer-readable program code, or instructions, embodied in or on the storage media. Any suitable computer readable storage media may be utilized, including hard disks, CD-ROMs, optical storage devices, magnetic storage devices, and/or any combination thereof.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. “Optional” or “optionally” means that the subsequently described event or circumstance may or may not occur, and that the description includes instances where the event occurs and instances where it does not.
Approximating language, as used herein throughout the specification and claims, may be applied to modify any quantitative representation that could permissibly vary without resulting in a change in the basic function to which it is related. Accordingly, a value modified by a term or terms, such as “about,” “approximately” and “substantially,” are not to be limited to the precise value specified. In at least some instances, the approximating language may correspond to the precision of an instrument for measuring the value. Here and throughout the specification and claims, range limitations may be combined and/or interchanged, such ranges are identified and include all the sub-ranges contained therein unless context or language indicates otherwise. “Approximately” as applied to a particular value of a range applies to both values, and unless otherwise dependent on the precision of the instrument measuring the value, may indicate +/−10% of the stated value(s).
The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. The embodiment was chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated.
The foregoing drawings show some of the processing associated according to several embodiments of this disclosure. In this regard, each drawing or block within a flow diagram of the drawings represents a process associated with embodiments of the method described. It should also be noted that in some alternative implementations, the acts noted in the drawings or blocks may occur out of the order noted in the figure or, for example, may in fact be executed substantially concurrently or in the reverse order, depending upon the act involved. Also, one of ordinary skill in the art will recognize that additional blocks that describe the processing may be added.
Number | Date | Country | |
---|---|---|---|
Parent | PCT/CN2022/076564 | Feb 2022 | US |
Child | 17652735 | US |