Video conferencing systems are becoming more popular in the United States and around the world. Many video conferencing systems are able to handle numerous incoming communication signals and are able to output a number of encoded signals for delivery to conference recipients. In one example, a party communicates audio and video signals to a number of users that have access to a video conferencing system. The video conferencing system ensures that the audio and video signals are in a format that can be consumed by the users.
Preparing and formatting audio and video signals for dissemination over a network often requires substantial processing resources. Before audio and video signals are sent over the network, the signals undergo an encoding process. This encoding process formats the signals so that they can be consumed by a receiving party. For example, an encoding process may evaluate a party's available network bandwidth and accordingly compress the signals so that they can be communicated over the network. Repeating this process for each recipient requires large processing resources and can limit a number of user conferences a conferencing system is able to handle.
Encoded signal reuse can increase the efficiency of a media server that receives and delivers media signals to users. The increase in efficiency may be achieved by reusing already-encoded media signals in favor of encoding a media signals before they are delivered to users. In one exemplary implementation, an evaluation process occurs before a signal is encoded. This evaluation process may include taking into consideration the composition of the signal, a network bandwidth that will be used to convey the signal once it is encoded, the capabilities of the device that will receive the signal after it is encoded, and a codec (compressor/decompressor) that would be used to encode the signal. If one or more of the evaluation parameters substantially match one or more parameters associated with an already-encoded signal, the already-encoded signal is selected in a favor of encoding the signal.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items.
Overview
The following disclosure describes an arrangement and/or method capable of reusing an already-encoded signal in favor of encoding a signal for dissemination to one or more users. In one implementation, an evaluation process occurs before a signal is encoded. This evaluation process may include taking into consideration the composition of the signal, a network bandwidth that will be used to convey the signal once it is encoded, the capabilities of the device that will receive the signal after it is encoded, and a codec that would be used to encode the signal. The foregoing are considered evaluation parameters that may be assessed before a signal is encoded. These evaluation parameters are compared against evaluation parameters belonging to already encoded signals. If the evaluation parameters of the signal to undergo encoding substantially match the evaluation parameters of an already encoded signal, the already encoded signal is selected and sent to a recipient. The original signal, which was to undergo encoding, is not encoded.
To distill the above, the various implementations described herein provide encoded-signal reuse. Encoded signal reuse has the potential to significantly reduce the processing requirements of a media server. This may increase a number of signals that the media server is able to process at any given moment.
An exemplary communication arrangement having a number of client computing devices in communication with a media server is described first. The description related to the communication arrangement is intended to provide context related to how a media server processes signals. The foregoing is followed by a description of an exemplary implementation of a media server and uses related thereto. Then, exemplary processes that encode signals or select encoded signals for communication to user computing devices are described. Finally, a general computing device is discussed.
An Exemplary Communication Arrangement
The media server 104 is designed to receive media signals, such as audio and video signals, and is able to communicate the received media signals to destination computing devices (e.g., the client computing devices 102). The phrases media signals or media signal, as used herein, are intended to refer to any media types, individually or collectively. These media types may include motion video, still video, audio, shared data, shared applications, and any other media in open or propriety standards that may be communicated over a network (e.g., the network 106).
The client computing devices 102 may represent computing devices used by users or subscribers during video conferences, or data conversations or conversations between devices or applications over a network (e.g., the network 106). Therefore, each of the client computing devices 102 may include a microphone and a speaker to capture and play audio signals. The client devices 102 may be further implemented with a camera and a display to capture and play video signals. The client computing devices 102 are illustrated as personal computers, but may also be implemented as other devices, such as a set-top box, a game console, a laptop computer, a portable digital assistant (PDA), a cellular phone, Voice over IP (VoIP) telephones, conventional phones, and so forth.
In one implementation, a client device 102 sends a media signal 108 over the network 106 to the media server 104. This media signal 108 is for dissemination to a plurality of other client devices 102. The media server 104 facilitates the dissemination to the plurality of other client devices 102 by ensuring that the media signal 108 is in a format that can be received by the other client devices 102. To that end, the media server 104 may evaluate an amount of network bandwidth available to each of the client devices 102 that are to receive the media signal 108. An amount of network bandwidth available to a given client device is considered one parameter that can be used when choosing a signal encoding procedure.
The media signal 108 may go through several transformations, such as decoding and encoding, in the following description. For clarity, the reference “media signal 108” will remain consistent.
The amount of network bandwidth available to a destination client device may determine how the media signal 108 is encoded. For example, those client devices 102 that have broadband network connections, such as DSL and cable modem connections, can receive media signals encoded using a low-compression codec (compressor/decompressor). But, those client devices 102 that have dialup network connections, might only be able to receive media signals encoded using a high-compression codec.
As is shown in
The signal module 112 is generally responsible for selecting codecs used to encode media signals. When possible, the signal module 112 will select already-encoded signals in favor of choosing to encode media signals. Already encoded signals are held in a buffer module 114. The buffer module 114 is part of the signal module 112; this is by way of example only.
The already encoded signals held in the buffer module 114 have associated parameters. These parameters relate to information that was considered before the already encoded signals were encoded. In this implementation, an amount of network bandwidth available to each of the client devices 102 that are to receive the media signal 108 are being considered. Thus, if the media signal 108 were to undergo encoding, the amount of bandwidth available to a particular destination client device 102 would be associated with the signal 108 during the encoding process. Other parameters may include the composition of a signal, the capabilities of the device that will receive a signal after it is encoded, and a codec that would be used to encode a signal.
To convey an aspect of the implementation, suppose that the buffer module 114 does not contain any already encoded signals. Accordingly, the media signal 108 needs to undergo encoding before it is sent to the client device 102. To that end, the signal module 112 evaluates the amount of bandwidth available to the client device 102 that will receive delivery of the media signal 108. Based on the evaluation, the signal module 112 chooses a codec to encode the media signal 108. A codec lookup table may be employed by the signal module 112 to facilitate the codec-selection process.
The signal module 112 passes the media signal 108, along with instructions indicating a codec that should be used to encode the media signal 108, to a sending module 116. The sending module 116 retrieves the indicated codec, encodes the media signal 108 and delivers the encoded media signal 108 to the client device 102.
The signal module 112 also sends the encoded media signal 108 back to the signal module 112 for storage in the buffer module 114. The encoded media signal 108 is packaged with at least one associated parameter that can be later referenced. In this case, the associated parameter is the amount of bandwidth available to the destination client device 102 that received the encoded media signal 108.
The media signal 108 is for dissemination to many client devices 102. Conventional media conveying systems may encode the media signal 108 before each communication to respective ones of the client devices 102. To do this requires a very large amount of processing overhead.
The media server 104 may convey media signals with greater efficiency. The buffer module 114 now includes the encoded media signal 108. Those destination client devices 102 that have access to an amount of bandwidth that is relatively close to the associated parameter linked to the encoded media signal 108 can likely consume the encoded signal 108. Therefore, instead of encoding the media signal 108 for those destination client devices 102, the signal module 112 passes the encoded media signal 108, retrieved from the buffer module 114, to the sending module 116 with instructions to disseminate the already encoded signal 108 to one or more client devices 102. Eliminating the encoding process in the sending module 116 may enhance the efficiency of the media server 104.
In one implementation, the media server 104 encodes a media signal two times. One encoded signal is sent to destination computing devices that have high-bandwidth connections, such as a connection that can handle a bit rate greater than 56 Kbps. The other encoded signal is sent to destination computing devices that have low-bandwidth connections, such as a connection that cannot handle bit rates greater than 56 Kbps. The two encoded signals are stored in the media server 104 and each encoded signal has a respective associated parameter.
The respective associated parameters simply indicate a level of network bandwidth a destination computing device needs to have in order to properly receive the encoded signal. For example, one associated parameter may indicate that the encoded signal may be sent to destination devices that have high bandwidth connections. A client device is considered to have a high bandwidth connection if the device can handle bit rates greater than 56 Kbps. A client device is considered to have a low bandwidth connection if the device cannot handle bit rates greater than 56 Kbps.
In another example implementation, the media server 104 encodes a media signal two times. One encoded signal is sent to destination computing devices that have high-bandwidth connections, such as a connection that can handle a bit rate greater than 56 Kbps. The other encoded signal is sent to destination computing devices that have low-bandwidth connections, such as a connection that cannot handle bit rates greater than 56 Kbps. The signal for the low bandwidth connections is encoded using a 20 Kbps codec. The two encoded signals are stored in the media server 104 and each encoded signal has a respective associated parameter.
The respective associated parameters simply indicate a level of network bandwidth a destination computing device needs to have in order to properly receive the encoded signal. For example, one associated parameter may indicate that the encoded signal may be sent to destination devices that have high bandwidth connections. A client device is considered to have a high bandwidth connection if the device can handle bit rates greater than 56 Kbps. A client device is considered to have a low bandwidth connection if the device cannot handle bit rates greater than 56 Kbps.
Using the foregoing principles, for each destination computing device, the media server 104 conducts a simple bandwidth connection evaluation, references the associated parameters of the encoded media signals, and sends an appropriate one of the two encoded media signals to the device. More than two media signal encodings may be produced to support a greater number of bandwidth ranges.
In another implementation, the media server 104 encodes a media signal one time and sends the encoded signal to all of the recipient destination devices. Here, the media server 104 may encode the media signal as if all of the destination devices have low bandwidth connections. Although this technique saves having to encode the same media signal multiple times, it may be at the expense of those destination devices that have high bandwidth connections.
The devices, modules and elements illustrated in
Exemplary Media Server Implementation
The Users 1-6 shown in the figure are participating in a collaborative video conference that may include one or more audio and video media signal exchanges between the Users. The media server 104 enables the conferencing functions of the collaborative video conference.
The receiving module 110 receives a User 1 signal and a User 2 signal. The signals may be audio or video media. If necessary, the User 1 and 2 signals are processed by a Real-Time Transport Protocol (RTP) module 200. As those skilled in the art appreciate, the RTP module 200 may be used to detect if there is any packet loss and to compensate for any delay jitter. The User 1 and 2 signals are passed to a decoder 204. The decoder 204 is configured to decode both audio and video media signals. The decoder 204 evaluates the protocol of the User 1 and 2 signals and decodes the signals appropriately. The decoded User 1 and 2 signals are in the form of decompressed linear samples.
The decompressed linear samples related to the User 1 and 2 signals are passed to a mixer module 206. The mixer module 206 is responsible for combining the User 1 and 2 signals. This is accomplished by simple addition of the decompressed linear samples. The result of this addition is a mixed linear stream (User 1&2 signal). The User 1&2 signal is sent to the signal module 112.
The signal module 112 is equipped with an evaluation module 208 and the buffer module 114. The evaluation module 208 is responsible for evaluating the capabilities of user computing devices interfaced with the media server 104. In some instances, the module 208 evaluates a network bandwidth available to a user's computing device (e.g., a speed of the network connection and/or packet loss over the connection) and the capabilities of the computing device (e.g., processing capabilities). The module 208 may also evaluate the composition of a received signal and may make an accounting of a codec used to encode a signal. The evaluation module uses these various evaluation techniques to determine if an already encoded signal stored in the buffer module 114 may be selected in favor of encoding a given signal.
As described, the evaluation module 208 now has the User 1&2 signal. The evaluation module 208 may reference the receiving module 110 to determine that the User 1&2 signal is composed of media signals received from User 1 and User 2, respectively. The evaluation module 208 is now familiar with the composition of the User 1&2 signal.
The User 3 is to take delivery of the User 1&2 signal. Thus, the evaluation module 208 evaluates an amount of network bandwidth available to User 3's computing device 102. The module 208 also evaluates the processing capability of User 3's computing device 102.
The evaluation module 208 now has a number of evaluation parameters that may be compared against parameters associated with already encoded signals held in the buffer module 114. In this case, the evaluation module 208 searches the buffer module 114 for an appropriate already encoded signal, but does not locate one.
The evaluation module 208 passes the User 1&2 signal to the sending module 116 for encoding. The User 1&2 signal has associated parameters that include the network bandwidth available to User 3's computing device 102, the capabilities of the computing device 102, the composition of the User 1&2 signal, and a codec reference. An encoder 210 selects the referenced codec from a codec repository (not shown) and encodes the User 1&2 signal. The encoded User 1&2 signal is processed by an RTP module 212 and a packetizer module 214, and delivered to the User 3.
The sending module 116 also bundles the encoded User 1&2 signal with the associated parameters that include the network bandwidth available to a User 3's computing device, the capabilities of the computing device, the composition of the User 1&2 signal, and a codec reference. The bundle is sent to the Buffer module 114 and stored there for possible reuse.
The media server 104 needs to ensure that the User 1&2 signal is delivered to the user computing devices 102 in a format that can be properly processed and consumed. The media server 104 has already accomplished this task for User 3's computing device 102 and is now ready to process the User 1&2 signal for delivery to User 4.
The evaluation module 208 evaluates the User 4's computing device 102 in the same manner described above in connection with the User 3's computing device 102. In particular, the evaluation module 208 evaluates an amount of network bandwidth available to User 4's computing device 102. The module 208 also evaluates the processing capability of User 4's computing device 102. The evaluation module 208 already knows, from the above, the composition of the User 1&2 signal.
The evaluation module 208 now has a number of evaluation parameters that may be compared against parameters associated with already encoded signals held in the buffer module 114. The evaluation module 208 searches the buffer module 114 for an appropriate already encoded signal and finds the encoded User 1&2 signal. The encoded User 1&2 signal is bundled with the associated parameters that include the network bandwidth available to User 3's computing device 102, the capabilities of the computing device 102, the composition of the User 1&2 signal, and a codec reference.
The evaluation module 208, based on a comparison of the evaluation parameters and the associated parameters bundled with the encoded User 1&2 signal, determines that the User 4 may receive the same encoded signal delivered to the User 3. The evaluation module 208 instructs the buffer module 114 to send the encoded User 1&2 signal to the sending module 116. The sending module 116 processes and sends the encoded User 1&2 signal to the User 4.
As can be understood from the foregoing, one encoding process was eliminated using the encoding-reuse principles described herein. Many more encoding processes may be eliminated as the described techniques are repeated for many destination client devices.
Packet loss may be measured using a special packet designed to get a response back from a destination computing device (e.g., a device 102), much like the echo of a sonar ping used to detect objects underwater. Such a special packet is often referred to as a “ping packet.” It is possible to send several (or even continuous) ping packets in succession over a network to a destination computing device. The number of responses returned from the destination computing device may be used to calculate an average packet loss of a current communication session.
Determining the capabilities of a computing device may be as simple as evaluating the raw processing power of a device's CPU. Such an evaluation will give a MIPS value or score. Higher MIPS values normally indicate higher performing computing devices. Other factors may also be used in determining a computing device's capabilities. These factors commonly include bus architecture, data bus width, and chip clocking values.
Measuring bandwidth available to a computing device can be fairly straightforward. One popular bandwidth measuring technique involves polling the technology facilitating communication with a network (e.g., the network 106). Polling of the facilitating technology generates feedback indicating a maximum bandwidth capability of the technology. Telephone modems, cable modems and DSL modems are a few examples of technologies that facilitate communication with a network.
To illustrate an exemplary use of the modules 502 and 504, reference is made back to
The buffer module 114 is also illustrated in
Exemplary Processes
For discussion purposes, the processes are described with reference to the communication arrangement 100 of
Referring to
At block 604, the media server 104 compares the evaluated one or more parameters with one or more parameters associated with an already encoded signal. Such an already encoded signal may be held in the buffer module 114, which is discussed in connection with the signal module 112. Recall, an already encoded signal may have an associated parameter that indicates a bandwidth level that the encoded signal is compatible with. In one implementation, the associated parameter is simply one of a high bandwidth or low bandwidth indicator. Therefore, at block 604, if a speed of the network connection is within the range 0-56 Kbps (low bandwidth), this evaluation parameter would match an associated parameter that is a low bandwidth indicator. Similarly, if a speed of the network connection is greater than 56 Kbps (high bandwidth), this evaluation parameter would match an associated parameter that is a high bandwidth indicator.
At block 606, if one or more of the evaluated parameters matches a parameter associated with an already encoded signal, the already encoded signal is selected. At block 608, the already encoded signal is sent to a destination computing device.
At block 610, if one or more of the evaluated parameters does not match a parameter associated an already encoded signal, a media signal undergoes encoding. At block 612, the encoded signal is saved for possible later use. At block 608, the encoded signal is sent to a destination computing device.
At block 708, a network bandwidth available to a destination computing device is evaluated. As discussed herein, other parameters may be evaluated as well. In one implementation, the evaluation module 208 processes the instructions of block 708.
At block 710, the media server 104 determines if there are any already encoded signals in a buffer (e.g., the buffer module 114). If not, at block 712, the media server encodes the received signal(s). At block 714, if already encoded signals are contained in the buffer, the already encoded signals are evaluated to see if one or more of the encoded signals has an associated parameter (e.g., a network bandwidth parameter) that matches the network bandwidth determined at block 708. If one or more encoded signals have a matching network bandwidth parameter, an encoded signal can be chosen if the composition of the encoded signal, before it underwent encoding, matches the received signal(s).
If an already encoded signal is found in the buffer module 114 (block 716), the encoded signal is selected at block 718. Otherwise, at block 712, the received signal is encoded.
Exemplary Computing Device
For the present implementations, the program modules 708 may include the various modules and elements of the media server 104. Other modules described herein may also be part of the program modules 908.
The computing device 900 may have additional features or functionality. For example, the computing device 900 may also include additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated in
The computing device 900 may also contain a communication connection 928 that allow the device to communicate with other computing devices 930, such as over a network like network 106 of
Various modules and techniques may be described herein in the general context of computer-executable instructions, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, and so forth. for performing particular tasks or implement particular abstract data types. These program modules and the like may be executed as native code or may be downloaded and executed, such as in a virtual machine or other just-in-time compilation execution environment. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments. An implementation of these modules and techniques may be stored on or transmitted across some form of computer readable media.
Conclusion
The above-described systems and methods attempt to improve signal encoding by reducing how often media signals are encoded. When encoded signals are reused, a video conferencing system, or other media signal processing technology, may enjoy significantly reduced processing overhead. Reduced processing overhead generally increases a video conferencing system's responsiveness. Although the systems and methods have been described in language specific to structural features and/or methodological acts, it is to be understood that the systems and methods defined in the appended claims are not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claimed system and method.