Patent Application
20150230002
The present disclosure pertains generally to cable television network technology, and particularly to adaptive and dynamic multiplexing techniques for interactive television services delivered over various network topologies including the Internet.
Interactive television services provide a television viewer with the ability to interact with their television in meaningful ways. Such services have been used, for example, to provide navigable menuing and ordering systems that are used to implement electronic program guides and pay-per-view or other on-demand program reservations and purchases, eliminating the need to phone the television provider. Other uses include interacting with television programming for more information on characters, plot, or actors, or interacting with television advertisements for more information on a product or for a discount coupon.
These services typically employ a software application that is executed on a server system located remotely from the TV viewer such as at a cable television headend. The output of the application is streamed to the viewer, typically in the form of an audio-visual MPEG Transport Stream. This enables the stream to be displayed on virtually any client device that has MPEG decoding capabilities such as a television set-top box. The client device allows the user to interact with the remote application by capturing keystrokes and passing these back to the application running on the server.
In cable system deployments, the headend server and its in-home set-top or other client are separated by a managed digital cable-TV network that uses well-known protocols such as ATSC or DVB-C. Here, “managed” means that any bandwidth resources required to provide these services may be reserved prior to use. Once resources are allocated, the bandwidth is guaranteed to be available, and the viewer is assured of receiving a high-quality interactive application experience.
In recent years, audio-visual consumer electronics devices increasingly support a Local Area Network (LAN) connection, giving rise to a new class of client devices: so-called “Broadband Connected Devices”, or BCDs. These devices may be used in systems other than the traditional cable television space, such as on the Internet. For example, a client device, such as a so-called smart TV, may implement a client application to deliver audio-visual applications streamed over a public data network from an audio-visual application streaming server to a television. A user may employ a remote control in conjunction with the client device to transmit interactive commands back to the application streaming server, thereby interacting with the server controlling the choice and delivery of desired content.
The “last mile” (the final leg of the telecommunications networks providing the actual connectivity to the end user) in public networks is typically made up of a number of network technologies, ranging from high-capacity fiber-optical networks to asymmetric digital subscription lines. In contrast inside a home, distribution is often realized by means of wireless technologies such as IEEE 802.11 networks (commonly known as Wi-Fi networks.) As a result, capacity (here meaning the maximum aggregate bandwidth a specific link is able to carry) varies between end-users, and due to the wireless technologies involved, capacity for a particular end-user also varies over time. Further, public data networks are not managed in the same way as private cable television distribution systems are. TCP, the most common transport protocol for the Internet, tries to maximize usage of its fair share of the capacity. As a result, it is impossible to guarantee a specific amount of bandwidth to applications running over such networks.
The intricacies of transmitting video over a network of varying capacity and available bandwidth (i.e., capacity not in use yet) conditions are a known challenge that has been successfully addressed. Examples of systems that transmit video over a network with varying capacity and available bandwidth (i.e., capacity not in use yet) include:
1. Video conference call systems,
2. Cloud game services,
3. HLS (HTTP Live Streaming), and
4. Progressive download video-on-demand.
Video conference call systems and cloud game services represent a type of system where a continuous low-delay video signal is encoded in real-time. The encoded stream adapts to changing network conditions by changing the picture quality, where a lower picture quality (typically realized by a higher average quantization of the coefficients that represent the picture) yields a lower average bitrate. Typically, these systems stream over an unreliable transport (such as UDP or RTP) and employ error correction and/or concealment mechanisms to compensate for loss. Any artifacts due to this loss or imperfect concealment are corrected over time due to the continuous nature of the signal. These systems require a complex and often proprietary client not only because of the complexity of the employed methods of concealment, but also because the client plays an important role in the measurement and reporting of the statistics that allow the server to make intelligent decisions about network conditions.
On the other end of the spectrum are systems that stream an offline-encoded, non-real-time stream over a reliable transport protocol like TCP/HTTP. These streams are progressively downloaded, where buffering makes the system robust for temporal variations in available bandwidth or capacity and, in the case of HLS for example, the stream changes to a different quality level depending on the capacity or sustained available bandwidth. In this case, the complexity of the client is relatively low and the components that make up the client are well-defined.
An interactive television service has a combination of properties of both of these previously mentioned types of systems. The streams exhibit low delay, real-time properties typically associated with UDP/RTP high-complexity, proprietary clients. However, the stream is received by relatively low-complexity clients using standard components. Typically such clients are more akin to progressive download clients using TCP/HTTP than to the clients that provide interactive or real-time services.
An interactive television service also has relatively static portions with a graphical user interface (GUI) that requires low-latency, artifact-free updates upon interactivity, combined with portions that have full motion video and audio that require smooth and uninterrupted play out.
Conventional systems do not adequately facilitate this combination of requirements. A new approach is therefore needed.
Digital television over a managed network such as a cable television system uses constant-bandwidth channels to carry multiple program streams. Multiplexing within a fixed allocation of bandwidth requires a multiplexer controller to manage the allocation of bandwidth among a group of competing program streams or competing sessions. In this manner, an individual program stream or session competes for bandwidth against the remainder of the program streams or sessions in the group of program streams or sessions. Control logic in the multiplexer controller manages the byte allocation among the program streams so that as few compromises as possible in quality are required and the compromises are evenly distributed among the group.
Managed networks form the vast majority of commercial television program distribution networks. However, video program consumption is rapidly moving to both live and on-demand consumption via the Internet, an unmanaged network. Today fully one-third of all Internet data traffic at primetime is from the popular Internet video service Netflix. In the near future, over 80% of all Internet traffic will be video data.
On an unmanaged network, such as the Internet, a single program stream (or session) competes for bandwidth from a large number of other unknown streams over which the multiplexer has no control. One of the many advantages of the systems and methods described herein is a multiplexer controller that can control sending video information over unmanaged networks and utilize a class-based, multi-dimensional control logic that optimizes the interactive user experience for interactive and on-demand television programming.
Interactive television services provide the viewer with the ability to interact with their television for the purposes of selecting certain television programming, requesting more information about the programming, or responding to offers, among many possible uses. Such services have been used, for example, to provide navigable menu and ordering systems that are used to implement electronic program guides and on-demand and pay-per-view program reservations. These services typically employ an application that is executed on a server located remotely from the viewer. Such servers may be, for example, located at a cable television headend. The output of a software application running on the server is streamed to the viewer, typically in the form of an audio-visual MPEG Transport Stream. This enables the stream to be displayed on virtually any client device that has MPEG decoding capabilities, including a “smart” television, television set-top box, game console, and various network-connected consumer electronics devices and mobile devices. The client device enables the user to interact with the remote application by capturing keystrokes and passing the keystrokes to the software application over a network connection.
An interactive television service combines the properties of both of the aforementioned types of systems (i.e., managed and unmanaged network topologies). Such services require low delay, perceptually real-time properties typically associated with Real Time Transport Protocol running over User Datagram Protocol (UDP/RTP) on high-complexity, proprietary clients. However, in interactive television applications the stream is received by relatively low-complexity clients using consumer-electronics-grade components. Typically, the clients are more akin to progressive download clients using Transmission Control Protocol/Hypertext Transfer Protocol (TCP/HTTP) than to the clients that typically provide interactive services.
An interactive television service is also a combination of relatively static image portions representing a graphical user interface (graphical UI or GUI) that requires low-latency, artifact-free updates responsive to user input, and other portions that may have video with associated audio that require smooth and uninterrupted play-out. Conventional multiplexers do not adequately facilitate this combination of data types over the Internet. For instance, with existing system that send data over the Internet, when large user interface graphics of a particular session need to be sent to a particular client, if unpredictable network congestions impacts delivery, such systems have no means available (except a drastic reduction in image quality) to scale back or modify the order of multiplex elements to allow a temporary large data block representing the UI graphics to pass, for just one example.
With an extraordinarily high number of sessions active across the Internet, the probability for disruption to video, audio and/or GUI data is certain. The only alternative that conventional systems have is for often drastic reductions in video quality or greatly lowering of frame rate or, worse, the interruption of program material while the receiving client device attempts to buffer sufficient data to proceed.
The present embodiments overcome these common obstacles to sending video programming and interactive television services over unmanaged networks to receiving client devices by exploiting class-based asset allocation. For example, improvement in video transmission across an unmanaged network is realized using multi-dimensional control loop-logic that is programmed to make the best choice in managing adverse network conditions by trading off latency with frame rate with video quality. Critical data such as audio is maximally protected against packet loss, which is desirable because “the ears don't blink”: audio interruptions are usually very objectionable compared to the same in video.
Furthermore, network latency is measured such that useful measures of network congestion can be estimated.
In some embodiments, a method of adapting content-stream bandwidth includes generating a content stream for transmission over an unmanaged network with varying capacity; sending the content stream, via the unmanaged network, toward a client device; monitoring the capacity of the unmanaged network; determining whether an aggregate bandwidth of an upcoming portion of the content stream fits the capacity, wherein the upcoming portion of the content stream corresponds to a respective frame time and includes video content and user-interface data; and, in response to a determination that the aggregate bandwidth does not fit the capacity, reducing a size of the upcoming portion of the content stream.
In some embodiments, a server system includes one or more processors and memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for performing the above-described method. In some embodiments, a non-transitory computer-readable storage medium stores one or more programs configured for execution by one or more processors of a server system. The one or more programs include instructions for performing the above-described method.
For a better understanding of the various described embodiments, reference should be made to the Detailed Description below, in conjunction with the following drawings. Like reference numerals refer to corresponding parts throughout the figures and description.
Reference will now be made to embodiments, examples of which are illustrated in the accompanying drawings. In the following description, numerous specific details are set forth in order to provide an understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.
In recent years, audio-visual consumer electronics devices increasingly support a Local Area Network (LAN) connection, giving rise to a new class of client devices: so-called “Broadband Connected Devices”, or BCDs. These devices may be used in systems other than traditional cable television, such as on the Internet. For example, a client device such as a smart TV may implement a client application to deliver audio-visual applications streamed over a public data network from an audio-visual application streaming server (also referred to as an application server) to a television. A user may employ a remote control in conjunction with the client device to transmit interactive commands back to the application streaming server, thereby controlling the content interactively.
Quality of service for the delivery of digital media over an unmanaged network is optimized by exploiting a class-based management scheme to control an adaptive bit-rate network multiplexer.
The compositor 103 composites fragments and video streams from various sources such as, but not limited to, the application engine 102, which generates fragments representing UI updates, and the transcoder 101, which transcodes video assets into composite-able assets. Feedback 109 from the proprietary TCP component 105, obtained through TCP's acknowledgment mechanism over the upstream channel 108, is used to determine a global frame time clock 104.
Conventional systems typically trade picture quality 203 for bitrate, but this does not yield satisfactory results in the disclosed system. The system of
Capacity (C),
Available bandwidth (A),
Average Delta One Way Delay (˜DOWD),
Round Trip Time (RTT), and
Loss rate.
Based on these inputs the control loop 409 calculates a frame rate, maximum chunk size, and pause signal that are provided as input to the application engine 402 and scheduler 403. For example, the frame rate is provided to the application engine 402, while the frame rate, maximum chunk size, and pause signal are provided to the scheduler 403.
In some embodiments, the application engine 402 uses the frame rate to adapt to variable bandwidth conditions. A reduction in frame rate by a factor 2 roughly yields a similar reduction in bit rate for an equivalent picture quality. The fragments from the application engine 402 may use a fixed quantization parameter to keep quality uniform across the interface. The output of the application engine 402 is therefore generally more peaky than that of a typical video asset because the fragments may use these fixed quantization parameters instead of being rate controlled.
In some embodiments, the transcoder 401 may have video assets in different frame rates flavors instead of quality levels. Video assets may be transcoded ahead of time and adaptability to various bandwidth conditions is achieved by transcoding a video asset in multiple bit-rate flavors (i.e., using multiple bit rates). In conventional systems, a reduction in bitrate is typically achieved by increasing the quantization of the coefficients that constitute the video frames. The result of this increase in quantization is a reduction in picture quality generally perceived as ringing, contouring, posterizing, aliasing, blockiness or any combination there-of, especially in scene changes. Instead of reducing the quality of the video and maintaining the frame rate, in some embodiments the frame rate is reduced and the quality maintained to achieve a similar reduction. The advantage is that for varying bandwidth conditions, the quality of the video remains the same albeit at a different frame rate. Another advantage is that by having a choice of frame rate options for video assets, the scheduler 403 can tradeoff UI latency for video frame rate.
In some embodiments, the transport component 413 employs an UDP-like streaming behavior with TCP semantics. The advantage of using the TCP protocol's semantics is that the client can run a standard TCP/HTTP protocol implementation. Using the TCP protocol also allows for easier traversal of NAT routers and firewalls that are often found in the last mile. The disadvantage of standard TCP is that it is generally not well suited for real-time, low-delay streaming because of its random back-off, fairness and retransmission properties. Therefore, in some embodiments the server system does not adhere to typical TCP behavior such as the slow start, congestion window, and random back-off, and instead sends segments in a way that suits the described real-time streaming requirements, while maintaining enough compliancy (such as following TCP's receive window and retransmission rules) to be able to use standard TCP client implementations.
The transport component 413 may have a send process 404 and a receive process 410. The send process 404 sends scheduled chunks as bursts of TCP segments, without regards to traditional TCP fairness rules, and retransmits lost segments as mandated by the TCP protocol upon loss indications from the receive process 410. The receive process 410 processes TCP acknowledgments (ACKs) and selective acknowledgments (SACKs) and timestamps pertaining to the segments sent by the send process 404. RFC 1323 describes TCP timestamps. In standard TCP implementations, the TCP timestamps are used in an algorithm known as Protection Against Wrapped Sequence numbers (PAWS). PAWS is used when the TCP window size exceeds the possible number of sequence numbers, typically in networks with a very high bandwidth/delay product. In some embodiments, the timestamps are used to determine server-side the link's capacity and available bandwidth by leveraging the fact that the burst transmission timespan can be compared to the client-side reception timespan. Conventional systems have algorithms that use these delta one way delays to derive the link's capacity and, by varying the exact timing of the segments in the burst, make an approximation of the available bandwidth. Instead of using special probe data to determine these statistics only at the start of a session, the server system uses the audio and video data itself to continuously measure changes in the link's capacity and available bandwidth by means of reading the time stamps of the returning TCP ACKs from the burst of TCP packets to the client. This measurement of return ACKs provides a means to determine network latency and congestion allowing for more accurate use of available bandwidth.
The same mechanisms can be implemented on top of standard UDP instead of TCP, assuming packet loss is handled by standard mechanisms such as Forward Error Correction or retransmissions.
An unmanaged network 414 (e.g., the Internet), is the environment over which the described system is designed to work. Such a network is typified by a plurality of downstream networks with queues and associated properties 405 and upstream networks with queues and associated properties 411. The downstream and upstream networks are generally asymmetrical with respect to capacity, available bandwidth and latency properties. The disclosed system assumes no prior knowledge over the properties of the unmanaged network, other than that variable latency, loss and reordering are assumed to occur. Some links, such as Wi-Fi links, may also exhibit temporary loss of all connectivity.
The client device running the client firmware 415 may be a broadband-connected set-top box, a broadband-connected television, a computer, or any other device. In some embodiments, the client device has a standard TCP client implementation 406, a thin client implementation 407, and an audio/video decoder 408 (e.g., that may decode MPEG-2, H.264/MPEG-AUDIO, AC3 or AAC video/audio streams).
In some embodiments, the audio/video decoder 408 is a hardware decoder. Typically, hardware decoders rely on a constant stream of audio/video data and do not handle buffer under-runs very well. Therefore, the thin client implementation 407 may implement methods to prevent under-run, such as the method of
The method of
The compositor 412 may generate transport streams with a system frame rate of 29.97 Hz for NTSC systems or 25 Hz for PAL systems. When the compositor 412 is said to change to another frame rate, it is the effective frame rate that may be changed. The effective frame rate is the rate with which the display changes, as opposed to the system frame rate, which is the rate at the frame clock advances. If the effective frame rate is lower than the system frame rate, the compositor may output intermediate null-frames in between frames that carry data that change the display. Suppose the system frame rate is 30 Hz and the effective frame rate is 15 Hz. In this case the compositor may output the following frames; E0-N1-E2-N3-E4-N5, where Et denotes an effective frame at system frame time t and Nt denotes a null frame at system frame time t. This can be arbitrarily extended to any effective frame rate (e.g., E0-N1-N2-E3-N4-N5 for 10 Hz and E0-N1-N2-N3-E4-N5 for 7.5 Hz).
The client firmware 415 may remove null-frames to compensate for earlier null-frames it introduced as instructed by the server. When the effective frame rate equals the system frame rate the stream may not have frames that can be removed. It is therefore advantageous to always have a system frame rate that is double the maximum effective frame rate. For a NTSC system the system frame rate may be 59.94 Hz and for PAL the system frame rate may be 50 Hz, although the maximum effective frame rate of transcoded assets is 29.97 Hz or 25 Hz respectively.
Another reason to use a system frame rate that is higher than the maximum effective frame rate may be to allow for more freedom in resampling video assets from film or cinema rate to the system frame rate. Conversely, converting assets from 29.97 Hz to 25 Hz and vice versa may yield better results when the system frame rate is higher and frames can be scheduled closer to their original frame time.
In some embodiments of the invention, the higher system frame rate may be used to separate video material from user interface material. This may be achieved server side by using the even frames for encoded video, while using the odd frames for composited user interface fragments (or vice versa). The advantages of this approach would be a reduced compositing overhead and the fact that the video may use encoding parameters that are incompatible with the fragment compositing process employed for the user interface fragments (For example, an embodiment that uses H.264 may use CABAC for encoding the video assets while using CAVLC for the composited fragments.), resulting in higher quality video.
In some embodiments of the invention, the concept of alternating video and user interface frames may also be used to retrieve and decode an out-of-band video asset. Additional benefit of such an approach is that for the video stream a progressive download of the asset can be used in combination with low latency server side encoded user interfaces. In some embodiments, the user interface and video share the same latency. It is not possible to send ahead video data without additional complexity on the client. If a system does send ahead video data, the client may be required to change timestamps to correct playback. However, tolerance with respect to varying link conditions would improve if the audio and video could be decoupled from the user interface and be buffered as in a normal progressive download system.
In some embodiments, this decoupling may be partially achieved by sending ahead audio. Discontinuities in audio playback are much more noticeable than temporary disruptions in video. The user experience is considerably enhanced if a few hundred milliseconds of audio were available to bridge temporary loss in connectivity. The system may send ahead audio, because audio and video can be synched via timestamps. At the client, audio-video sync may be achieved by matching audio timestamps with video timestamps. Therefore, it is not a problem to send ahead audio up to a certain amount. This way, a certain degree of connectivity robustness and a continuous user experience is achieved without a latency penalty, which would otherwise spoil the interactive experience for the user.
In the event of a temporary disruption of link connectivity, the audio and video may become out of sync because the audio keeps playing while the video buffer may under-run. To alleviate this problem, the thin client 407 may use a null-frame stuffing/removing mechanism as has been described.
Audio may also be sent ahead over a separate logical or physical connection.
As has been described, the compositor 412 may use frame rate and latency instead of, or in addition to, picture quality to adapt the audio/video streams and user interface to the available bandwidth. Adapting by changing the quantization of the coefficients is well-known. Adaptability using latency and/or frame rate is best described by example.
In some embodiments, an interactive application includes a user interface with a partial-screen video stream.
Now suppose that the aggregate bandwidth does not fit (i.e., exceeds the maximum chunk size) because, for example, a user interface update at t is too big to fit the budget. Audio is typically a fixed component and the user experience benefits from uninterrupted audio playback. Therefore a policy decision has to be made whether to give precedence to the user interface update or the video stream.
If the user interface update is the result of the user interacting with the system, it may be assumed that low latency of the response is more important than maintaining video frame rate. An example of how this works out is depicted in
If the user interface update is not the result of the user interacting but, for example, is application-timer induced, it may be assumed that the user is watching the video stream and it may be beneficial for the user experience to maintain the video frame rate and delay the user interface. Such a scenario is depicted in
The examples of
Video frames may be sent ahead because the video streams may be pre-transcoded or real-time transcoded video assets that have been transcoded at least a few frames ahead of time. The structure of a typical multi-frame-rate video asset is depicted in
An advantage of reducing frame rate instead of reducing picture quality is that frames at a particular time t are more or less equivalent; they represent roughly the same picture. Especially for scene changes this is a considerable advantage because it is typically at scene changes that blocking artifacts become very apparent. As has been noted before, a reduction in frame rate by 2 yields a reduction in bitrate by 2 for an equivalent picture quality. It should be noted, though, that equivalent frames may not be identical for different frame rates. Due to the intricacies of the block based encoder and its quantization process, the exact pixel values depend on the exact sequence of intra and inter prediction and quantization process. Switching from one frame rate to another may introduce small errors, albeit much smaller than when switching between streams of different quality. An appropriate intra refresh process may be applied to alleviate a buildup of these small errors.
The concept of the effective frame rate is also used by the transport. As has been outlined in
Assume Bapp (in bits per second) is the bit rate at which an application is specified to work full frame rate, with system frame rate Fs (in frames per second). Then the following may hold:
MaxChunkSize=(Bapp/Fs)/8
If the available bandwidth or capacity exceeds Bapp, the effective frame rate Fe may be equal to Fs. Or, half that of Fs if system is to benefit from the advantages outlined before. If the available bandwidth or capacity is less than Bapp, the control loop may decide to either shrink the MaxChunkSize or change the effective frame rate to the next available divider. In some embodiments, the effective frame rate may be changed. The advantages of maintaining the bit budget for individual frames and reducing the frame rate have been outlined for picture quality, but the advantage also extends to the transport; by reducing frame rate instead of picture quality, the average amount of data per frame remains the same for varying bitrates. For efficiency reasons it is advantageous to always send the data in the chunk using the maximum TCP segment size. Since the transport derives statistics per segment, reducing the amount of data would reduce the amount of segments over which statistics can be derived. Unless, of course, the segment size is reduced.
Maintaining a relatively high number of segments from which to derive statistics is important because clients may have limited TCP timestamp properties. RFC 1323 does not specify the units in which the timestamps are expressed, nor the resolution of its updates. Tests have shown that common timestamp granularity (the resolution at which different segments can be distinguished from each other) range from one millisecond up to ten. A typical TCP segment for a typical Internet connection to the home may carry approximately 1450 bytes of data. A typical Bapp setting for BCD sessions may be for example 6 Mbps, at which a TCP segment takes roughly 2 milliseconds. (Assuming that the link's capacity roughly equals Bapp.) A timer granularity of 10 milliseconds roughly equates to 5 segments, which is not enough to directly derive any useful statistics.
In the disclosed system, the transport 413 increases accuracy of the measurements by building a histogram of arrival times. Suppose a client has a timestamp granularity of 10 milliseconds. The first segment in a frame marks the first histogram slot 0. The timestamps of any subsequent segments are subtracted by the timestamp of this first slot, adding to the histogram's slot 0, 1, . . . , n. Note that the arrival of the first segment is typically not synchronized with the slot timing. Therefore, a typical histogram for 12 segments may look like (where # denotes a segment):
0: ###
1: ######
2: ###
3:
Histograms like these may be used to derive a number of network properties and conditions, some of which are specified below.
If the departure constellation (the intervals between the segments) of the segments was sufficiently dense, that is, the segments were transmitted as a burst with minimal inter segment intervals, the capacity of the narrow link, the link with the lowest capacity, may be derived from the slot with the largest number of hits.
If the width of the histogram of a dense departure constellation exceeds the number of slots expected within an effective frame time (four for NTSC at 30 frames per second, because the first and last slot are arbitrarily aligned to the arrival constellation), the stream may either be exceeding the capacity:
0: ##
1: ###
2: ###
3: ###
4: #
or may be experiencing intermittent network problems (such as latency due to Wi-Fi interference):
0: ###
1: ######
2: #
3:
4: ##
If the width of the histogram is lean (using only the first 2 slot), the system is not using the full capacity of the link:
0: ###########
1: #
2:
3:
4:
The histogram approach may be used even if the client allows for a better timestamp granularity by artificially limiting the granularity to a granularity around 10-millisecond. For example, a common granularity of 4 milliseconds may be changed to 8 milliseconds slot times.
Artificially limiting the granularity may also be used to scale the algorithm to lower effective frame rates. For example, when the effective frame rate is halved, the granularity may be halved as well (e.g., from 10 milliseconds to 20 milliseconds). This effectively scales the algorithm with the effective frame rate; all proportions including average amount of data and number of segments, picture quality, semantics of histogram, and others remain the same while only the effective frame rate changes. Note, however, that if more accuracy is available, histograms can be arbitrarily recalculated to see whether a ‘magnified’ histogram yields more information. Timestamps may also be used directly, if granularity permits.
If the timestamp granularity is too low, RTT (round trip time) may be used as an alternative with the disadvantage that variations in upstream congestion may skew the results.
Throughout the disclosure, references have been made to capacity and available bandwidth. Capacity, from an end-to-end perspective, is the maximum aggregate bandwidth the narrow link is able to carry, where narrow link is the link with the lowest capacity. Available bandwidth, from the same perspective, is the unused capacity. Overflowing the capacity should be avoided, while overflowing the available bandwidth is a policy decision. The system continuously measures capacity and estimates an effective frame rate that will fit the capacity. By sending chunks as tightly spaced TCP segments (or bursts), the system is able to capture its share of the capacity and minimize latency. Coexistence with unmanaged protocols such as unmodified TCP may be achieved by the fact that interactive applications have a strong variable bit rate (VBR) profile and hardly ever fully use the MaxChunkSize. Any additional knowledge about available bandwidth may enhance the decision to either maintain the current effective frame rate or reduce it.
In addition to on-demand feature files, more and more live cable television programming is moving to the Internet in addition to cable and satellite distribution. Internet-delivered (unmanaged network delivered) content is typically received via the equivalent of a cable or satellite set-top box. An example of this type of receiver is the processing capability built into contemporary smart TVs where, in addition to a standard TV tuner, the system of
Typically, network-connected set-top boxes have components similar to those as are summarized in
The functionality described herein may be embodied in many different forms, including, but in no way limited to, computer program logic for use with a processor (e.g., a microprocessor, microcontroller, digital signal processor, or general purpose computer), programmable logic for use with a programmable logic device (e.g., a Field Programmable Gate Array (FPGA) or other PLD), discrete components, integrated circuitry (e.g., an Application Specific Integrated Circuit (ASIC)), or any other means including any combination thereof
Computer program logic implementing all or part of the functionality previously described herein may be embodied in various forms, including, but in no way limited to, a source code form, a computer executable form, and various intermediate forms (e.g., forms generated by an assembler, compiler, linker, or locator). Source code may include a series of computer program instructions implemented in any of various programming languages (e.g., an object code, an assembly language, or a high-level language such as Fortran, C, C++, JAVA, or HTML) for use with various operating systems or operating environments. The source code may define and use various data structures and communication messages. The source code may be in a computer executable form (e.g., via an interpreter), or the source code may be converted (e.g., via a translator, assembler, or compiler) into a computer executable form.
The computer program may be fixed in any form (e.g., source code form, computer executable form, or an intermediate form) either permanently or transitorily in a tangible storage medium, such as a semiconductor memory device (e.g., a RAM, ROM, PROM, EEPROM, or Flash-Programmable RAM), a magnetic memory device (e.g., a diskette or fixed disk), an optical memory device (e.g., a CD-ROM), a PC card (e.g., PCMCIA card), or other memory device. The computer program may be fixed in any form in a signal that is transmittable to a computer using any of various communication technologies, including, but in no way limited to, analog technologies, digital technologies, optical technologies, wireless technologies (e.g., Bluetooth), networking technologies, and internetworking technologies. The computer program may be distributed in any form as a removable storage medium with accompanying printed or electronic documentation (e.g., shrink wrapped software), preloaded with a computer system (e.g., on system ROM or fixed disk), or distributed from a server or electronic bulletin board over the communication system (e.g., the Internet or World Wide Web).
Hardware logic (including programmable logic for use with a programmable logic device) implementing all or part of the functionality previously described herein may be designed using traditional manual methods, or may be designed, captured, simulated, or documented electronically using various tools, such as Computer Aided Design (CAD), a hardware description language (e.g., VHDL or AHDL), or a PLD programming language (e.g., PALASM, ABEL, or CUPL).
Programmable logic may be fixed either permanently or transitorily in a tangible storage medium, such as a semiconductor memory device (e.g., a RAM, ROM, PROM, EEPROM, or Flash-Programmable RAM), a magnetic memory device (e.g., a diskette or fixed disk), an optical memory device (e.g., a CD-ROM), or other memory device. The programmable logic may be fixed in a signal that is transmittable to a computer using any of various communication technologies, including, but in no way limited to, analog technologies, digital technologies, optical technologies, wireless technologies (e.g., Bluetooth), networking technologies, and internetworking technologies. The programmable logic may be distributed as a removable storage medium with accompanying printed or electronic documentation (e.g., shrink wrapped software), preloaded with a computer system (e.g., on system ROM or fixed disk), or distributed from a server or electronic bulletin board over the communication system (e.g., the Internet or World Wide Web).
The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the scope of the claims to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen in order to best explain the principles underlying the claims and their practical applications, to thereby enable others skilled in the art to best use the embodiments with various modifications as are suited to the particular uses contemplated.
This application claims priority to U.S. Provisional Patent Application No. 61/984,703, titled “Class-Based Intelligent Multiplexing over Unmanaged Networks,” filed Apr. 25, 2014, and is a continuation-in-part of U.S. patent application Ser. No. 13/438,617, titled “Reduction of Latency in Video Distribution Networks using Adaptive Bit Rates,” filed Apr. 3, 2012, both of which are incorporated by reference herein in their entirety.
| Number | Date | Country | |
|---|---|---|---|
| 61984703 | Apr 2014 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13438617 | Apr 2012 | US |
| Child | 14696463 | US |
