Patent Application
20130219073
The present invention relates generally to data communication, and more particularly to communication of display streams.
Video information is typically encoded prior to communication of the information over a network. The video information can be encoded in any one of a number of different formats, depending on the device that generates or stores the video information. In order to allow the video information to be processed by a video sink device, such as a display device, a transcoder device can be employed to translate the encoded video to a format that is understood by the display device.
To illustrate, in one embodiment the transcoding device can concurrently generate three different display streams, designated for purposes of discussion as stream A, stream B, and stream C. Each of the display streams has a different bitrate, with stream A having the highest bitrate, stream B the next highest bitrate, and stream C the lowest bitrate. The transcoding device can determine that the bandwidth of the network is above a defined threshold and, in response, communicate stream A to the network for transmission to the sink device. The transcoding device continues to generate stream B and stream C as stream A is communicated to the network. In response to determining that the network bandwidth is below the defined threshold, the transcoding device stops communicating stream A to the network, selects stream B, and communicates stream B to the network for transmission to the sink device. As network conditions continue to change, the transcoding device can continue to switch among the streams. For example, if the bandwidth increases above the threshold, the transcoding device can return to communicating stream A. If the bandwidth falls further below the threshold, the transcoding device can switch to communicating stream C. Further, the transcoding device can synchronize switching between the streams so that a user viewing the media information at the sink device does not experience lag, skipping, or other distracting effects.
Referring
The wide area network (WAN) 102 is a communication network generally configured to provide data generated or stored at devices in the WAN 102. In one embodiment, the WAN 102 is the Internet. The media data provided by the WAN 102 can be television programming, recorded or stored video, movies, video clips, and the like, or any combination thereof, and can include video information, audio information, and other multimedia information.
Both the WAN 102 and the media source 101 can provide media data in accordance with one or more defined formats, such as a format in compliance with a Motion Picture Experts Group (MPEG) standard. Accordingly, the media data can be provided in a stream format, whereby the stream includes the media data in compressed form. A stream of media data (which can include video data, audio data, or other multimedia data, or a combination thereof) targeted for display, or targeted for storage and subsequent display, is referred to herein as a display stream. Each display stream is associated with a bitrate, which is the number of bits of information communicated by the display stream per defined time unit (e.g. per second).
The transcoding device 110 is a device generally configured to receive an input stream and transcode the input stream to produce a plurality of display streams. Accordingly, the transcoding device 110 can be incorporated in or be a portion of any of a number of media devices, including a set top box, DVD or other media player, computer server or other computer device, cell phone or other telecommunication device, and the like. In an embodiment, the transcoding device 110 transcodes the input streams by changing an encoding format of the received streams and by changing the data rate (referred to herein as transrating) the received streams. In another embodiment the transcoding device 110 maintains the encoding format of the received streams but transrates the streams.
As described further herein, the transcoding device 110 is configured to concurrently generate multiple display streams for a selected input stream, such that each of the generated display streams has different transmission characteristics. As used herein, a transmission characteristic refers to a feature of a display stream that affects the speed with which the display stream can be communicated by a given network. Examples of transmission characteristics include the data rate of a display stream, the resolution of media data associated with the display stream, and the like. Thus, in one embodiment the transcoding device 110 can generate, based on a particular received display stream, multiple display streams where each of the generated display streams has a different bitrate. In another embodiment each of the generated display streams can have a different resolution. For example, one of the generated display streams can be associated with high definition (HD) video information while another of the generated display streams can be associated with standard definition (SD) video information. In still another embodiment, the transmissible display streams can vary across multiple transmission characteristic types. Thus, each of the generated display streams can have different bit rates and have different resolutions. Further, although each of the generated display streams has different transmission characteristics, each represents the same media data as provided by the input stream. Thus, for example, if the input stream represents a particular television program, each of the generated display streams will represent that same television program.
The network 120 is a communication network generally configured to provide a communication backbone for communication of display streams and other information. Accordingly, the network 120 can be a local area network, a wide area network, and the like. The media sinks 130, 140, and 150 each represent a different device generally configured to receive display streams, to translate the received display stream to a format suitable for rendering at a display device, and to display the rendered data at the display device. Accordingly, the media sinks 130, 140, and 150 can each be a computer device, television device, set top box, cell phone or other telecommunication device, and the like, or a combination thereof.
In operation, at least one of the content sinks 130, 140, or 150 can request, via the network 120, media information from one of media source 101 and wide area network 101. In response, the requested media data, encoded in a display stream, is provided to the transcoding device 110. In response, the transcoding device 110 generates multiple display streams based on the received input stream, with each of the generated display streams having different transmission characteristics. The transcoding device 110 selects among the generated display streams based upon one or more detected characteristics of the network 120, and communicates the selected display stream to the network 120 for transmission to the sink that requested the media information. Further, the transcoding device 110 can dynamically switch between the generated display streams in response to changes in the characteristics of the network 120. This can be better understood with reference to
At time 201 the transcoding device 110 determines a network characteristic of the network 120. For purposes of illustration, it is assumed that the network characteristic is a bandwidth of the network 120. In other embodiments, the network characteristic can be communication latency, a quality of service, and the like. At time 201 the transcoding device 110 determines that the network bandwidth falls within a defined range, such that bitrate associated with stream B is predicted to provide the best user experience. To illustrate, the range can be defined such that, if stream A were to be selected when the bandwidth is within the range, the user would experience lag, stuttering, or other undesirable artifacts when viewing the media information. Further, if stream C were to be selected, the user would experience an undesirable delay in commencement of media playback without a commensurate increase in quality of the playback relative to stream B. Accordingly, the transcoder 110 selects stream B and communicates stream B to network 120 between time 201 and time 205.
At time 205 the transcoder 110 determines that the network bandwidth has increased, such that the bandwidth is above the range associated with stream B. In response, the transcoder 110 selects stream A, stops communication of stream B to the network 120, and begins communication of stream A to the network 120. The change from communication of stream B to communication of stream A is synchronized so that the user does not experience skipping or other undesirable artifacts of the switch.
For example, in one embodiment each of the streams A, B, and C are composed of segments, such as packets, whereby each segment includes one or more frames of media information. Further, each segment and include a header or other identifying information that includes a time stamp associated with the media information. Accordingly, each of the streams A, B, and C are generated such that a common timestamp between the streams is associated with the same media information. Therefore, in response to determining that the stream being communicated is to be changed, the transcoder 110 can determine the timestamp of the last segment of the selected stream that was communicated to the network 120, locate the segment of the next stream to be communicated that has the same timestamp (or the next timestamp in sequence), and begin communication of the stream with the selected segment.
For example, the transcoder 110 can determine, in switching from stream A to stream B, that the last segment of stream A that was communicated to the network 120 was associated with timestamp value 00100. The transcoder 110 can locate the segment of stream B that has timestamp value 00100, or the segment of stream B that has the timestamp value next in sequence (e.g. 00101), and begin communication of stream B with the selected segment.
In another embodiment, the transcoder 110 can synchronize the change in streams based on the timing associated with communication of selected control information. For example, for some encoding formats a control header is periodically included in the stream. The control header can include, for example, information about the bit rate, resolution, or other characteristics of the stream. Accordingly the transcoder 110 can synchronize the change in streams such that the first segment communicated by the newly-selected stream is a segment that includes the control header. This reduces the likelihood that the change in streams will result in decoding errors or other errors at the sink device.
Returning to
In the illustrated example of
In another embodiment, the transcoder 110 can generate different sets of streams depending upon the network characteristics. This can be better understood with reference to
At time 301 the transcoding device 110 determines a bandwidth of the network 120. At time 301 the transcoding device 110 determines that the network bandwidth falls within a defined range associated with stream B. Accordingly, the transcoder 110 selects stream B and communicates stream B to network 120 between time 301 and time 305.
At time 305 the transcoder 110 determines that the network bandwidth has decreased, such that the bandwidth is below the range associated with stream B. In response, the transcoder 110 selects stream C, stops communication of stream C to the network 120, and begins communication of stream C to the network 120. The change from communication of stream B to communication of stream C is synchronized so that the user does not experience skipping or other undesirable artifacts of the switch.
In addition, in response to the decrease in network bandwidth, the transcoder 110 stops generation of stream A, and begins generation of stream D, so that transcoder 110 concurrently generates streams B, C, and D. In one embodiment, stream D has a lower bit rate than stream C. In another embodiment, stream D has the same bit rate as stream C, but the media data associated with stream D has a lower resolution than the resolution associated with stream C. In still another embodiment, stream D can have both a lower bit rate and lower resolution relative to stream C.
At time 310 the transcoder 110 determines that the network bandwidth has decreased, such that the bandwidth is below a range associated with stream C. In response, the transcoder 110 selects stream D, stops communication of stream C to the network 120, and begins communication of stream D to the network 120. Thus, by beginning generation of stream D at time 305, prior to the reduction in bandwidth at time 310, the transcoder 110 is able to switch to stream D more quickly than if it began generation of stream D at time 310. This can reduce latency, transmission errors, or other difficulties arising from the reduction in network bandwidth.
At time 315, the transcoder 110 determines that the network bandwidth has increased, such that the bandwidth is within the range associated with stream B. In response, the transcoder 110 selects stream B, stops communication of stream D to the network 120, and begins communication of stream B to the network 120. In addition, the transcoder 110 stops generation of stream D and returns to generation of stream A, such that the transcoder is prepared to switch to stream A if the network bandwidth further increases.
Referring to
The processor 400 is configured to detect network characteristics of the network 120. In one embodiment, the processor 400 determines the network characteristics, such as bandwidth, based on feedback data received from the network interface controller 495. For example, the network interface controller 495 can include one or more measurement devices to determine the network bandwidth, latency, and other network characteristics. Based on the detected network characteristics, the processor 400 can designate one of the stored generated streams for communication to the network 120. In response, the selected stream is transmitted according to a direct memory access (DMA) method via external system input/output (IO) bus 420 past north bridge 405 into the host memory (host DRAM 310). Processor 400, using a timer driven dispatcher, at an appropriate time, will route the selected stream stored in host DRAM 410 to network interface controller 495, which then routes the stream to the network 120.
One implementation of the invention is as sets of computer readable instructions resident in the random access memory of one or more processing systems configured generally as described in
In the preceding detailed description of the figures, reference has been made to the accompanying drawings which form a part thereof, and in which is shown by way of illustration specific preferred embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, and it is to be understood that other embodiments may be utilized and that logical, mechanical, chemical and electrical changes may be made without departing from the spirit or scope of the invention. To avoid detail not necessary to enable those skilled in the art to practice the invention, the description may omit certain information known to those skilled in the art. Furthermore, many other varied embodiments that incorporate the teachings of the invention may be easily constructed by those skilled in the art. Accordingly, the present invention is not intended to be limited to the specific form set forth herein, but on the contrary, it is intended to cover such alternatives, modifications, and equivalents, as can be reasonably included within the spirit and scope of the invention. The preceding detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 201210039852.2 | Feb 2012 | CN | national |
