The present disclosure relates generally to network conferencing. In one example embodiment, the disclosure relates to adaptive rate of screenshot capture in screen sharing.
Network conferencing refers to interactive communication technologies that allow participants at two or more locations to interact. Such network conferencing may be achieved, for example, by way of real-time two-way video and audio transmissions. In a network conference, a participant may communicate and share data with other participants. For example, a participant can transmit images or contents rendered on his video display to other participants in the network conference. Such sharing of display is referred to as screen sharing. With screen sharing, participants within the network conference may see the display or screen viewed by other participants.
It should be appreciated that screen sharing can be computationally intensive and requires a large amount of processor resources. As a result, screen sharing during a network conference can adversely affect the performance of other applications executing on the same computing device that is processing the screen sharing. Furthermore, the transmission of data used in screen sharing may require a large amount of network bandwidth, which may further delay the transmission of other data.
The present disclosure is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which:
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of an example embodiment of the present disclosure. It will be evident, however, to one skilled in the art that the present disclosure may be practiced without these specific details.
The example embodiments described herein provide various techniques for adjusting a rate of screenshot capture used in screen sharing. In general, this rate of screenshot capture corresponds to a frame rate at which the display is animated. The rate of screenshot capture may be adjusted in reference to a variety of different parameters. For example, as explained in more detail below, the rate may be adjusted in reference to a usage of a processor. In another example, as explained in more detail below, the rate may be adjusted in reference to a compression ratio. As a result, the rate of screenshot capture is not constant and is instead, adaptive to various fluctuating parameters, such as usage of a processor and compression ratio.
Screen sharing uses two-way video transmissions between endpoints 104 and 108 to share rendered display. As depicted in
The processor monitoring module 206 is configured to monitor a usage of a processor. The usage indicates an amount of workload that is handled by a processor. That is, the usage may indicate the amount of resources that is used by the processor. The amount may be expressed as a percentage. For example, a load of 100% indicates that processor is fully utilized, whereas a load of 50% indicates that the processor is sitting idle half the time. To monitor the usage, the processor monitoring module 206 may itself calculate the usage. Alternatively, the processor monitoring module 206 may periodically retrieve or receive the usage from a variety of third-party applications that are configured to calculate the usage. It should be appreciated that a variety of processors may be monitored, such as central processing units (CPU), graphics processing units (GPU), and other processors.
The screen sharing module 208 is configured to capture screenshots for transmittal to other computing devices when screen sharing in a network conference. A “screenshot,” as used herein, is a reproduction or copy of a display rendered on a video display, which may be saved as a graphics file. The screen sharing module 208 captures the screenshots at a certain rate. As explained in more detail below, this screen sharing module 208 is also configured to adaptively adjust the rate of screenshot capture in reference to various parameters, such as a usage of the processor received from processor monitoring module 206 and/or a compression ratio derived from file sizes of compressed screenshots.
The video sharing module 210 is configured to interface with a video camera, and to receive video captured by the video camera. Such video may be directly transmitted to other computing devices in the network conference. Alternatively, the video may be transmitted to screen sharing module 208, which may capture the video for transmission to other computing devices. The coder-decoder (codec) module 212 is configured to encode and/or decode screenshots captured by screen sharing module 208. An example of a codec 212 is a video codec that is configured to compress and/or decompress video data. In video compression, a quantity of data used to represent video images can be reduced by image compression and motion compensation. Examples of codec module 212 include International Telecommunications Union (ITU) H.261 codec, ITU H.263 codec, MPEG-4 codec, Sorenson Video Codec, and other codecs.
The network conference module 207 may also include the network protocol module 214 that is configured to encode the compressed screenshots for transmittal over a computer network. An example of a network conference module 207 is a Transmission Control Protocol (TCP)/Internet Protocol (IP) encoder. It should be appreciated that in other example embodiments, computing device 200 may include fewer or more modules apart from those shown in
As depicted in
This rate of screenshot capture is adaptive and may be adjusted at 304. In an example embodiment, this rate may be adjusted in reference to the usage of the processor. That is, the number of screenshots captured within a time interval, which is the rate of screenshot capture, may be defined in reference to the usage of the processor. Adjustments may be made by adjusting the number of screenshots and/or adjusting the time interval. In an example, the rate may be increased by increasing the number of screenshot captured. Alternatively, the rate may be increased by decreasing the time interval. In another example, the rate may be decreased by decreasing the number of screenshots captured. Alternatively, the rate may be decreased by increasing the time interval.
As explained in more detail below, the rate of screenshot capture may be adjusted or defined in reference to a comparison between the usage of the processor with one or more thresholds. The screenshots are then transmitted at 306 to another computing device that is included in the network conference system.
New Rate=Max(Current Rate−1, Min_Rate) (1.0)
where the current rate of screenshot capture is incrementally subtracted by one screenshot with every adjustment until the minimum rate is reached.
On the other hand, if the usage does not exceed the upper threshold, then another comparison is made at 408. Here, the usage of the processor is compared to a lower threshold. This lower threshold is less than the upper threshold discussed above. An example of a lower threshold may be defined as 30% of full usage. In another example, the lower threshold may be defined as 10% of full usage. If the usage is less than or falls below the lower threshold, then the processor is not heavily loaded and may be able to handle a high rate of screenshot capture. As a result, the rate of screenshot capture is increased at 416. Similarly, the rate of screenshot capture may be increased in increments until a maximum rate of screenshot capture is reached. For example, the rate may be adjusted to or defined as:
New Rate=Min(Current Rate+1, Max_Rate) (2.0)
where the current rate of screenshot capture is incrementally added by one screenshot with every adjustment until the maximum rate is reached.
It should be appreciated that two thresholds are used in the comparison because two thresholds prevent frequent adjustments by providing a range between the upper threshold and lower threshold where adjustments are not made. However, in another example embodiment, one threshold may be used in the comparison. Here, the rate of screenshot capture is decreased if the usage exceeds this one threshold. Vice versa, the rate of screenshot captured may be increased if the usage falls below the same threshold. As a result, it should be appreciated that a comparison can be made with one or more thresholds.
Still referring to
The rate of screenshot capture, in accordance with another example embodiment, can also be adjusted at 504 in reference to the compression ratio. As explained in more detail below, this compression ratio may identify a type of display being captured and the rate of screenshot capture may be adjusted to optimize the quality of the display being shared. Additionally, the rate of screenshot capture may be increased or decreased based on comparison with one or more thresholds, as discussed in more detail below. The screenshots may then be captured at the adjusted rate at 506 and compressed at 508. The compressed screenshots may then be transmitted to another computing device included in the network conference system at 510.
It should be appreciated that a compression ratio may identify the type of display included in the captured screenshot. A graphical display is a type of display that is mostly comprised of images. Examples of graphical displays include displays of pictures, animation, drawings, and videos. Another example of a type of display is a text display that is mostly comprised of text. Examples of text displays include displays of documents, spreadsheets, and presentations. It should be noted that different types of displays may have different compression ratios. That is, some types of displays may be more or less compressible than other types of displays. For example, a codec specifically configured to encode video images may be able to more efficiently compress a graphical display than a text display. That is, a video codec may compress a screenshot of a graphical display into a smaller file than compared to a compression of a screenshot of a text display. The compression ratio may therefore be used to identify a type of display that is compressed. The adjustment of the rate of screenshot capture in reference to the compression ratio may therefore correspond to an adjustment in reference to the type of display being compressed. As a result, in an example embodiment, the rate of screenshot capture can be adjusted in reference to the type of display being compressed.
As depicted in
The compression ratio is then compared to an upper threshold at 708. In this example embodiment, this upper threshold is defined relative to the compression ratio. For example, the upper threshold may be defined as 10% compressibility (or 90% incompressibility). In another example, the upper threshold may be defined as 25% compressibility (or 75% incompressibility). Since the compression ratio is the reduced file size recorded after compression divided by the uncompressed file size recorded before compression, a high compression ratio indicates that the screenshot is not highly compressible, which can identify that the screenshot is comprised mostly of text display. Therefore, if the compression ratio exceeds the upper threshold, then the rate of screenshot capture is decreased at 710 because a text display does not usually change as much as a graphical display. In an example, the rate of screenshot capture may be decreased in accordance with Equation 1.0, as defined above. With the decreased rate, the quality of the captured screenshot may be increased. That is, with less screenshots being captured, the resolution of the screenshot may be increased such that the resulting text display is sharper and clearer than a screenshot captured with a lower resolution, but at a higher rate of screenshot capture.
On the other hand, if the compression ratio does not exceed the upper threshold, then another comparison is made at 712 with a lower threshold. In this example embodiment, this lower threshold is defined relative to the compression ratio and is less than the upper threshold. For example, the lower threshold may be defined as 90% compressibility (or 10% incompressibility). In another example, the lower threshold may be defined as 75% compressibility (or 25% incompressibility). In this example, a low compression ratio indicates that the screenshot is highly compressible, which can identify that the screenshot is comprised mostly of graphical display. Therefore, if the compression ratio falls below the lower threshold, then the rate of screenshot capture is increased at 714. Again, in an example, the rate of screenshot capture may be increased in accordance with Equation 2.0, as defined above. That is, a high rate of screenshot capture can result in smoother animation shown during screen sharing, but the screenshot is captured at a lower resolution, which results in less clarity when compared with a screenshot captured at a higher resolution.
It should be noted that the comparison of the compression ratio with the upper and lower threshold depends on the convention used to define the compression ratio. As an example, this compression ratio may also be defined as the uncompressed file size recorded before compression divided by the reduced file size recorded after compression. In this example, a high compression ratio indicates that the screenshot is highly compressible while a low ratio indicates that the screenshot is not highly compressible. As a result, if the compression ratio exceeds the upper threshold, then the rate of screenshot capture is increased. On the other hand, if the compression ratio falls below the lower threshold, then the rate of screenshot capture is decreased.
Still referring to
It should be appreciated that in addition to adjusting the rate of screenshot capture in reference to either the usage of the processor or the compression ratio, the rate of screenshot capture may also be adjusted in reference to a combination of the usage of the processor and the compression ratio. In an example embodiment, the compression ratio is identified and the usage of the processor is monitored. The rate of screenshot capture may then be adjusted in reference to both the compression ratio and the usage of the processor. As an example, if the usage of the processor exceeds a certain upper threshold and the compression ratio exceeds another upper threshold, then the rate of screenshot capture may be decreased. On the other hand, if the usage of the processor falls below a certain lower threshold and the compression ratio falls below another lower threshold, then the rate of screenshot capture may be increased.
The example computing device 800 includes processor 802 (e.g., a central processing unit (CPU), a graphics processing unit (GPU) or both), main memory 804 and static memory 806, which communicate with each other via bus 808. Computing device 800 may further include video display unit 810 (e.g., a plasma display, a liquid crystal display (LCD) or a cathode ray tube (CRT)). Computing device 800 also includes alphanumeric input device 812 (e.g., a keyboard), user interface (UI) navigation device 814 (e.g., a mouse), disk drive unit 816, signal generation device 818 (e.g., a speaker) and network interface device 820.
The disk drive unit 816 includes machine-readable medium 822 on which is stored one or more sets of instructions and data structures (e.g., software 824) embodying or utilized by any one or more of the methodologies or functions described herein. Software 824 may also reside, completely or at least partially, within main memory 804 and/or within processor 802 during execution thereof by computing device 800, main memory 804 and processor 802 also constituting machine-readable, tangible media.
The software 824 may further be transmitted or received over network 826 via network interface device 820 utilizing any one of a number of well-known transfer protocols (e.g., HTTP).
While machine-readable medium 822 is shown in an example embodiment to be a single medium, the term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term “machine-readable medium” shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present application, or that is capable of storing, encoding or carrying data structures utilized by or associated with such a set of instructions. The term “machine-readable medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical and magnetic media, and carrier wave signals.
While the invention(s) is (are) described with reference to various implementations and exploitations, it will be understood that these embodiments are illustrative and that the scope of the invention(s) is not limited to them. In general, techniques for adaptively adjusting a rate of screen capture may be implemented with facilities consistent with any hardware system or hardware systems defined herein. Many variations, modifications, additions, and improvements are possible.
Plural instances may be provided for components, operations or structures described herein as a single instance. Finally, boundaries between various components, operations, and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the invention(s). In general, structures and functionality presented as separate components in the exemplary configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the invention(s).