Patent Application
20160191961
This disclosure relates generally to transcoding video data, and specifically to fragmented video transcoding systems and methods.
The prevalence of streaming video on Internet Protocol (IP) networks has led to the development of Adaptive Bitrate (ABR) (e.g., HTTP-based) streaming protocols for video. There are multiple different instantiations of these protocols. However, ABR streaming protocols typically involve a video stream being broken into short, several-second-long encoded fragment files that are downloaded by a client and played sequentially to form a seamless video view, with the video content fragments being encoded at different bitrates and resolutions (e.g., as “profiles”) to provide several versions of each fragment. Additionally, a manifest file can typically be used to identify the fragments and to provide information to the client as to the various available profiles to enable the client to select which fragments to download based on local conditions (e.g., available download bandwidth). For example, the client may start downloading fragments at low resolution and low bandwidth, and then switch to downloading fragments from higher bandwidth profiles to provide a fast “tune-in” and subsequent better video quality experience to the client.
One example includes a fragmented video transcoding system. The system includes a video fragmenter configured to receive a linear input video data feed and to generate a plurality of video input fragment files corresponding to separate portions of the linear input video data feed. The system also includes a transcoder system configured to encode the plurality of video fragment files to generate a plurality of transcoded output video fragment files to be accessible for delivery to at least one client device
Another example includes a method for transcoding a video data stream. The method includes generating a plurality of video fragment files corresponding to separate portions of a received linear input video data feed and storing the plurality of video fragment files in a video fragment storage. The method also includes encoding the plurality of video fragment files via a plurality of transcoders to generate a plurality of transcoded video fragment files and storing the plurality of transcoded video fragment files in a transcoded fragment storage. The method further includes generating a video delivery manifest corresponding to metadata associated with the plurality of transcoded video fragment files via a playlist builder to facilitate video streaming to at least one client device.
Another example includes a video ecosystem. The system includes a fragmented video transcoding system. The fragmented video transcoding system includes a video fragmenter configured to receive a linear input video data feed and to generate a plurality of video fragment files corresponding to separate portions of the linear input video data feed. The fragmented video transcoding system also includes a transcoder system comprising a plurality of transcoders that are configured to concurrently encode a set of the plurality of video fragment files in a time-staggered manner to generate a plurality of transcoded video fragment files sequentially and uninterrupted in real-time. The system further includes a video delivery system configured to provide video streaming of the plurality of transcoded video fragment files to at least one client device in response to a request for video content corresponding to the plurality of transcoded video fragment files.
This disclosure relates generally to transcoding video data, and specifically to fragmented video transcoding systems and methods. The fragmented video transcoding can be implemented in a video ecosystem, such as to provide storage and/or delivery of video data to a plurality of client devices, such as via a video streaming service. As used herein, the term video data is intended to encompass video, audio and video, or a combination of audio, video and related metadata. As an example, the fragmented video transcoding system can include one or more video fragmenters configured to receive a linear input video feed and to generate a plurality of video fragment files that correspond to separate chunks of the linear input video data feed. As an example, each input video fragment file can include overlapping portions of adjacent (e.g., preceding and/or subsequent) video fragment files of the input video feed. The fragmented video transcoding system can also include a transcoder system that is configured to encode the plurality of video fragment files into a plurality of transcoded video fragment files that can be stored in a transcoded fragment storage. The transcoded video fragment files thus can be accessible for storage and/or delivery to one or more client device, which delivery can include real time streaming of the transcoded video or storage in an origin server for on-demand retrieval.
In some examples, the transcoder system can include a plurality of transcoders. For instance, the different transcoders can employ different encoding algorithms (e.g., combining spatial and motion compensation) to transcode the video fragment files and provide corresponding transcoded video fragment files in each of the different encoding formats, such as associated with different bitrates and/or resolution. Additionally or alternatively, a plurality of different transcoder subsystems can each include a plurality of transcoders for encoding the video to a respective encoding format (e.g., representing the input video in compressed format). The transcoders in a given transcoder subsystem can be implemented to concurrently encode the video fragment files in a time-staggered, parallel manner to provide the plurality of transcoded video fragment files sequentially and uninterrupted in real-time. Accordingly, each transcoder subsystem can provide transcoded video fragment files in real-time, even if implementing a longer-than-real-time encoding scheme. In addition, the transcoder system can implement a variety of different techniques for aligning audio and video in each of the transcoded video fragment files based on the encoding of the individual video fragment files rather than a linear input video data feed.
The video ecosystem 10 includes a fragmented video transcoding system 14. The fragmented video transcoding system 14 is configured to convert a linear input video data feed, demonstrated in the example of
As used herein, the term “video fragment file” refers to a chunk of media data feed V_FD that is one of a sequence of video chunks that collectively in a prescribed sequence corresponds to the linear input video. For example, the fragmenter can produce each video fragment file to include a fragment (e.g., a multi-second snippet) of video in its original resolution and format. In other examples, the video fragment files can include snippets of video in other formats (e.g., encoded video). The video fragment files can be stored in video fragment storage (e.g., non-transitory machine readable medium) as snippets of the digital baseband video versions (e.g., YUV format) of the linear input video data feed, can be stored as transport stream (TS) files, or can be stored as having a duration that is longer than the resultant video fragments that are provided to the client device(s) 12, as described herein. As an example, each video fragment file can be a data file that includes just video, both video and audio data, or can refer to two separate files corresponding to a video file and an audio file to be synchronized.
The transcoder system 18 is configured to encode (e.g., transcode) the plurality of video fragment files from its original format into corresponding transcoded video fragment files in one or more encoded output video formats. The transcoded video fragment files can be stored in a transcoded fragment storage, such that they are accessible for video streaming to the client device(s) 12 in one or more desired formats. For example, the video delivery system 20 can be implemented an origin server for storing the transcoded video or via a respective video streaming service to deliver streaming media to the clients 12.
As an example, the transcoder system 18 can include a plurality of transcoders, such that different transcoders can be implemented in different transcoder subsystems to employ different encoding protocols to provide the transcoded video fragment files in the different protocols, such as associated with different bitrates and/or resolutions. Thus, the transcoder system 18 can generate multiple different transcoded video fragment files for each video fragment file, with each of the different transcoded video fragment files for a given video fragment file encoded to a different bitrate to accommodate a range of bitrates for use in adaptive bitrate (ABR) streaming. Additionally or alternatively, each of the different transcoded video fragment files for a given video fragment file can be encoded to a different video encoding format packaged in a container for streaming (e.g., HTTP Live Streaming (HLS), HTTP adaptive streaming (HAS), Adobe Systems HTTP dynamic streaming, Microsoft smooth streaming, MPEG dynamic adaptive streaming over HTTP (DASH) or other ABR protocols) and at multiple bitrates to accommodate different ABR technologies.
As another example, each of transcoder subsystems of the transcoder system 18 can include multiple transcoders that are implemented to concurrently encode the video fragment files in a time-staggered manner in parallel to provide the plurality of transcoded video fragment files sequentially and uninterrupted in real-time. In this approach, the video ecosystem 10 can provide video streaming of the transcoded video fragment files in real-time, even with a longer-than-real-time encoding scheme of the transcoder system 18.
The transcoded video fragment files can thus be stored in memory for subsequent delivery (e.g., in an origin server) or be delivered more immediately to the client devices 12 via a video delivery system 20. As described herein, the term “video delivery” and “delivery” with respect to video data refers to any of a variety of manners of providing video data to one or more client devices (e.g., one or more of the client devices 12), such as including broadcast, multicast, unicast, video streaming, or any other way to transmit video data. As an example, the video delivery system 20 can access the transcoded video fragment files from a storage device, such as in response to a request for video content from one of the client devices 12, and can provide the transcoded video fragment files as the requested video content to the respective client device(s) 12. For example, the video delivery system 20 can be configured as or can include an HTTP server, such as to provide linear or on-demand delivery of the transcoded video fragment files. As another example, the video delivery system 20 can be configured as or can include a TS streaming engine to assemble the transcoded video fragment files and broadcast the transcoded video fragment files as a data stream to the client devices 12, such as based on any of a variety of Internet Protocol (IP) broadcast techniques.
Therefore, the video ecosystem 10 can provide video delivery in both linear and nDVR ecosystems of the linear input video data feed V_FD, in either ABR or fixed bitrate streaming protocols. The video ecosystem accomplishes this by fragmenting the linear input video data feed V_FD prior to encoding the video data via the transcoder system 18. As a result of fragmenting prior to encoding, the video ecosystem 10 can implement longer-than-real-time transcoders in a linear environment, such as to provide high computation and/or high-video quality (e.g., multi-pass or high-complexity high-efficiency video coding (HEVC)). The fragmentation of the linear input video data feed V_FD prior to the encoding also allows a more simple ABR packaging methodology based on the transcoder system 18 encoding the video fragment files, as opposed to the linear input video data feed V_FD itself. Additionally, by implementing a large number of transcoders in the transcoder subsystems of the transcoder system 18, the video ecosystem 10 can allow for an unlimited number of instantaneous decoder refresh (IDR) aligned video fragments without requiring any communication between transcoders, which facilitates aligning transcoded fragments downstream. Furthermore, the file-to-file transcoding of (e.g., from fragmented files to transcoded files) is more error resilient and mitigates potential failures. This can result in increased video quality at playout. The flexible arrangement of the transcoders in the transcoder system 18 can also enable the addition of new codecs (e.g., HEVC) through the introduction of different encoders into the transcoder system 18 without having to alter input or output interfaces.
The fragmented video transcoding system 50 includes one or more video fragmenters 52 that are each configured to receive the linear input video data feed V_FD. As mentioned, the linear input video data feed V_FD can be uncompressed video or be in another video format (e.g., MPEG-2, H.264 H.265 or the like). The video fragmenter(s) 52 are configured to generate a plurality of video fragment files, demonstrated in the example of
The VFFs 54 stored in the video fragment storage 56 can each be arranged as files of video fragments and corresponding audio fragments having a duration of one or more seconds of time (e.g., less than one minute, such as about 3-5 seconds). The VFFs 54 can correspond to any of a variety of different formats of video fragment files. For example, the VFFs 54 can be stored as baseband versions (e.g., YUV format) of the linear input video data feed V_FD. As an example, additional metadata, such as audio and data Program Identification files (PIDs) can also be stored using an encapsulation format to preserve timing relationships between the video and audio portions of the VFFs 54. As another example, the VFFs 54 can be stored as TS files, such as to allow audio and other PID data to be stored in the same file with associated synchronization information. Alternatively, if the linear input video data feed V_FD is provided in an MPEG-2 or H.264 format, for example, each of the VFFs can be generated as having a closed Group of Pictures (GOP) data structure that begins with an I-frame.
Additionally or alternatively, each VFFs 54 can be generated to include overlapping portions of media that are redundant with a portion of adjacent fragments, for example overlapping with its immediately preceding VFFs and its immediately subsequent VFF. Thus, the VFFs 54 can have a duration that is longer than the resultant video fragments that are provided to the client device(s) 12, and the video fragmenter(s) 52 can provide data that specifies the frames of the VFFs 54 are to be transcoded, as described in greater detail herein. To enable use of overlapping VFFs 54 in the transcoder system 50, information is provided with each VFF (e.g., metadata included with the VFF or separately signaled) to specify which frames of a given VFF are to be transcoded in the output.
The fragmented video transcoding system 50 also includes a transcoder system 58 that is configured to encode the VFFs 54 into transcoded video fragment files that correspond to the VFFs 54, demonstrated in the example of
In the example of
As a further example, each of the transcoders 64 can be configured to implement high-quality multi-pass encoding, which can implement a longer-than-real-time encoding scheme. For instance, a set of multiple parallel transcoders 64 in each of transcoder subsystems of the transcoder system 58 can be implemented to concurrently encode the VFFs 54 in a time-staggered manner to output each of the plurality of transcoded video fragment files sequentially and uninterrupted in real-time. Thus, the parallel transcoding provides for streaming of the TVFFs 60 in real-time, even when each of the transcoders implements a longer-than-real-time encoding scheme to generate the TVFFs 60.
The fragmented video transcoding system 50 also includes a playlist builder 66 to generate a manifest that defines properties for the TVFFs in each respective encoded output stream. In the example of
By way of further example, the playlist builder 66 is configured to monitor a drop folder (e.g., a specified resource location) associated with the transcoded fragment storage 62 for new TVFFs 60 that are generated by the transcoder system 58 and to automatically create a new video delivery manifest file in response to the storage of the TVFFs 60 in the transcoded fragment storage 62. As an example, the video delivery manifest can include information about the most recently available TVFFs 60, such as including encoding formats, file size, bitrates, resolutions, and/or other metadata related to the TVFFs 60. As another example, the playlist builder 66 can extract the time duration of the TVFFs 60 to include the time duration in the video delivery manifest(s), such as by extracting the time duration data from the transcoder system 58 or by extracting the time duration data directly from the VFFs 54 or the TVFFs 60.
The video delivery manifest(s) can then be provided to a directory (e.g., stored in the transcoded fragment storage 62) that is accessible by the video delivery system 20. Thus, the video delivery system 20 can serve the video delivery manifest(s) to the client devices 12 in response to a request for video content from the respective client devices 12. Accordingly, the video delivery manifest(s) can identify the available TVFFs 60 to the client device 12 to enable the client device 12 to request TVFFs 60 associated with a desired video content at a bitrate and/or resolution that is based on an available bandwidth to provide the appropriate video quality possible according to ABR streaming technology implemented at the client device.
The transcoder system 100 includes a plurality X of transcoder subsystems 102, with X being a positive integer. Each of the transcoder subsystems 102 can correspond to a separate respective encoding protocol that can be implemented to encode the video fragment files (e.g., the VFFs 54) to generate aligned transcoded video fragment files (e.g., the TVFFs 60). Each of the TVFFs can be aligned according to alignment information (e.g., an IDR access unit) that is provided in each TVFF. Each of the transcoder subsystems 102 includes a plurality of transcoders 104, demonstrated as pluralities Y and Z in the example of
As an example, each of the transcoder subsystems 102 can be associated with different encoding protocols and be configured to encode the TVFFs to respective bitrates and resolutions. The protocols can include, but are not limited to H.264, MPEG-2, and/or HEVC encoding formats. Thus, each of the transcoder subsystems 102 can generate multiple different transcoded video fragment files corresponding to each of the video fragment files. Each of the transcoded video fragment files generated from a given one of the transcoder subsystems 102 thus can be provided in a different video coding format for the each of the video fragment files. As an example, each of the transcoders 104 can be configured to implement high-quality multi-pass encoding, and can provide multiple transcoded video fragment files for each of the video fragment files that are each encoded at different bitrates and/or resolutions. Thus, the transcoded video fragment files can provide the same or greater level of video quality as other coding formats while having increased compression due to the video encoding technique.
As another example, the plural transcoders 104 in each of transcoder subsystems 102 of the transcoder system 100 can implement longer-than-real-time encoding in a linear encoding environment (e.g., for each video fragment file—VFFs 54) by adding overall latency and transcoding different video fragment files concurrently in parallel in a time-staggered manner.
By way of further example, at a time T0, the first transcoder 152 begins to encode a first video fragment file VFF1. At a time T1, the second transcoder 154 begins to encode a second video fragment file VFF2, while the first transcoder 152 continues to encode the first video fragment file VFF1. At a time T2, the third transcoder 152 begins to encode a third video fragment file VFF3, while the first transcoder 152 continues to encode the first video fragment file VFF1 and the second transcoder 154 continues to encode the second video fragment file VFF2. At a time T3, the fourth transcoder 158 begins to encode a fourth video fragment file VFF4, while the first transcoder 152 continues to encode the first video fragment file VFF1, the second transcoder 154 continues to encode the second video fragment file VFF2, and the third transcoder 156 continues to encode the third video fragment file VFF3.
At a time T4, the first transcoder 152 finishes encoding the first video fragment file VFF1, and thus provides a corresponding first transcoded video fragment file V1. Thus, at the time T4, the first transcoded video fragment file V1 can begin being streamed to one or more associated client device 12 that requested the corresponding video content. Also at the time T4, the first transcoder 152 begins to encode a fifth video fragment file VFF5, while the second transcoder 154 continues to encode the second video fragment file VFF2, the third transcoder 156 continues to encode the third video fragment file VFF3, and the fourth transcoder 158 continues to encode the fourth video fragment file VFF4.
At a time T5, the second transcoder 154 finishes encoding the second video fragment file VFF2, and thus provides a corresponding second transcoded video fragment file V2. Thus, at the time T5, the second transcoded video fragment file V2 can be streamed to the associated client device 12 immediately following the first transcoded video fragment file V1 in real-time, and thus uninterrupted to the user of the client device 12. Also at the time T5, the second transcoder 154 begins to encode a sixth video fragment file VFF6, while the third transcoder 156 continues to encode the third video fragment file VFF3, the fourth transcoder 158 continues to encode the fourth video fragment file VFF4, and the first transcoder 152 continues to encode the fifth video fragment file VFF5.
At a time T6, the third transcoder 156 finishes encoding the third video fragment file VFF3, and thus provides a corresponding third transcoded video fragment file V3. Thus, at the time T6, the third transcoded video fragment file V3 can be streamed to the associated client device 12 immediately following the second transcoded video fragment file V2 in real-time, and thus uninterrupted to the user of the client device 12. Also at the time T6, the third transcoder 156 begins to encode a seventh video fragment file VFF7, while the fourth transcoder 158 continues to encode the fourth video fragment file VFF4, the first transcoder 152 continues to encode the fifth video fragment file VFF5, and the second transcoder 154 continues to encode the sixth video fragment file VFF6.
At a time T7, the fourth transcoder 158 finishes encoding the fourth video fragment file VFF4, and thus provides a corresponding fourth transcoded video fragment file V4. Thus, at the time T7, the fourth transcoded video fragment file V4 can be streamed to the associated client device 12 immediately following the third transcoded video fragment file V3 in real-time, and thus uninterrupted to the user of the client device 12. Also at the time T7, the fourth transcoder 158 begins to encode an eighth video fragment file VFF8, while the first transcoder 152 continues to encode the fifth video fragment file VFF5, the second transcoder 154 continues to encode the sixth video fragment file VFF6, and the third transcoder 156 continues to encode the seventh video fragment file VFF7.
It is understood that the timing diagram 150 continues therefrom to demonstrate the encoding of additional subsequent video fragment files VFFs into corresponding transcoded video fragment files Vs that immediately follow the preceding transcoded video fragment files Vs in real-time. Accordingly, by fragmenting the linear input video data feed V_FD prior to the transcoder system 100, the transcoder system 100 can concurrently encode a set of the video fragment files VFFs in a time-staggered and segmented manner to provide the corresponding transcoded video fragment files Vs sequentially and uninterrupted in real-time.
The timing diagram 150 of
Referring back to the example of
The fragmented video transcoding system 200 includes a video fragmenter 202 configured to generate video fragment files 204, demonstrated in the example of
In the example of
The transcoder system 206 is thus configured to encode the non-overlapping portions of the video fragment files 204 to generate the respective transcoded video fragment files 208. In the example of
In the example of
The fragmented video transcoding system 250 also includes a transcoder system 260 that is configured to encode the video fragment files 254 from an input format (corresponding to the linear input feed) to an output format corresponding to transcoded video fragment files 262 that correspond to the video fragment files 254. As demonstrated in the example of
Therefore, similar to as described previously in the example of
In view of the foregoing structural and functional features described above, a method in accordance with various aspects of the present invention will be better appreciated with reference to
What have been described above are examples. It is, of course, not possible to describe every conceivable combination of components or methodologies, but one of ordinary skill in the art will recognize that many further combinations and permutations are possible. Accordingly, the disclosure is intended to embrace all such alterations, modifications, and variations that fall within the scope of this application, including the appended claims. As used herein, the term “includes” means includes but not limited to, the term “including” means including but not limited to. The term “based on” means based at least in part on. Additionally, where the disclosure or claims recite “a,” “an,” “a first,” or “another” element, or the equivalent thereof, it should be interpreted to include one or more than one such element, neither requiring nor excluding two or more such elements.
This application claims the benefit of U.S. Provisional Application No. 62/098,395, filed Dec. 31, 2014, and entitled FRAGMENTED-BASED LINEAR TRANSCODING SYSTEMS AND METHODS, which is incorporated herein in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 62098395 | Dec 2014 | US |
