The invention relates to a method of segmenting at least two received content streams, and a streaming server performing the method.
The invention further relates to a master server and a method performed at the master server of aligning created segments of received content streams with segments of the received content streams created by a slave server
Moreover, the invention relates to a slave server and a method performed at the slave server of aligning created segments of received content streams with segments of the received content streams created by a master server.
When streaming content over a network, a technique known as Adaptive Bitrate Streaming (ABS) is commonly used. In ABS, the same content to be streamed is encoded at different bitrates and segmented into data sets known as segments.
In other implementations, a video stream may be provided at different resolutions, or an audio stream may be provided in different languages, which streams need to be segmented as in the example of ABS.
Thus, in the example of ABS, the segments comprise the content at different streaming quality levels depending on the bitrate at which the content initially was encoded.
A client can subsequently switch between the different segments; if the client has capacity to stream a higher-quality segment, it can request to do so whereas if the capacity of the client should decrease, it can switch to a lower-quality segment. Typically, the client reads a so called manifest file stored at a streaming server, where the different segments and their associated quality is identified.
In ABS, the segments of different quality levels must correspond to the same piece of the content, to allow a client to switch between segments without experiencing a gap in the rendered content or a content overlap. This requires segmentation to be aligned across all quality levels available to a streaming server supplying the client with the content.
For redundancy, the same content streams are typically sent to multiple streaming servers. In such a scenario, it is then crucial that the segmentation and identification of segments are aligned across the multiple streaming servers, to allow clients to perform failover to another server during a streaming session.
An object of the present invention is to solve, or at least mitigate, these problems in the art and to provide aligning of segments pertaining to separate content streams, as well as aligning of segments over multiple servers.
This object is attained in a first aspect of the invention by a method performed at a streaming server of segmenting at least two received content streams. The method comprises identifying a current boundary point for each of the at least two received content streams indicating where the respective content stream can be segmented, determining whether an identifier for a segment is included in a segment information database at the streaming server, the identified segment having a starting point corresponding to a timestamp of the identified current boundary point, and if so segmenting the received content stream, for which the segment identifier was included in the segment information database, at the timestamp and including the segment identifier with the segmented stream.
This object is attained in a second aspect of the invention by a streaming server configured to segment at least two received content streams. The streaming server comprises a processing unit arranged to cause the streaming server to be operative to identify a current boundary point for each of the at least two received content streams indicating where the respective content stream can be segmented, determine whether an identifier for a segment is included in a segment information database at the streaming server, the identified segment having a starting point corresponding to a timestamp of the identified current boundary point, and if so to segment the received content stream, for which the segment identifier was included in the segment information database, at the timestamp and including the segment identifier with the segmented stream.
Advantageously, by using a boundary point of a first received content stream as reference for segmenting the first content stream at subsequent boundary points, as well as for segmenting further received content streams at encountered boundary points, the created segments of the received streams of the streaming server are aligned.
In an embodiment of the invention, one of the at least two received content streams is advantageously initially segmented at a starting timestamp for creating a segment with which the other received content streams can be synchronized.
In a further embodiment, in case no identified segment in the segment information database has a starting point corresponding to a timestamp of the identified current boundary point, the streaming server determines whether the timestamp of the identified current boundary point indicates an allowable segment length, and if so segments the received content stream, for which the segment length is indicated to be allowable, at the timestamp and including the segment identifier with the segmented stream, and updating the segment information database. Advantageously, a predefined allowable length is set, which the predefined length the streaming server will account for when performing the segmenting of received content streams.
In an embodiment, the streaming server performs the updating of the segment information database by adding an incremented segment identifier and setting a starting point of the segment corresponding to the incremented identifier to the time stamp of the identified current boundary.
Advantageously, the segment information database is continuously updated as new segments are created on the basis of encountered boundary points, thereby maintaining the alignment of the segments pertaining to the different received content streams.
In still another embodiment, when the streaming server determines whether the timestamp of the identified current boundary point indicates an allowable segment length, the server further determines whether a difference in time between the time stamp of the identified current boundary point and a starting point of a closest previously identified segment exceeds a minimum allowable segment length, and if not precludes the time stamp of the identified current boundary point as a possible segmenting point. Advantageously, if the streaming server concludes that the difference in timestamp does not exceed a minimum length, the identified current boundary point is skipped as possible segmenting point.
In a further embodiment, in case the difference in time between the timestamp of the identified current boundary point and a starting point of a closest previous identified segment exceeds a minimum allowable segment length, the streaming server determines whether a difference in time between the timestamp of the identified boundary point and a starting point of a closest previously identified segment is less than a maximum allowable segment length, and if not resynchronizes to one of the received content streams. Advantageously, if the streaming server concludes that the difference in timestamp exceeds a maximum length, the boundary point where segmenting should have been performed has passed, and the segments are no longer aligned; the streaming server will thus resynchronize to a content stream and start over with creating a new segment information database.
This object is attained in a third aspect of the invention by a method performed at a master server of aligning created segments of received content streams with segments of received content streams created by a slave server. The method comprises providing the slave server with a current segment information database comprising at least an identifier for a segment and a starting point for each identified segment.
This object is attained in a fourth aspect of the invention by a master server configured to align created segments of received content streams with segments of received content streams created by a slave server. The master server comprises a processing unit arranged to cause the master server to be operative to provide the slave server with a current segment information database comprising at least an identifier for a segment and a starting point for each identified segment.
This object is attained in a fifth aspect of the invention by a method performed at a slave server of aligning created segments of received content streams with segments of the received content streams created by a master server. The method comprises acquiring, from the master server, a current segment information database comprising at least an identifier for a segment and a starting point for each identified segment.
This object is attained in a sixth aspect of the invention by a slave server configured to align created segments of received content streams with segments of the received content streams created by a master server. The slave server comprises a processing unit arranged to cause the slave server to be operative to acquire, from the master server, a current segment information database comprising at least an identifier for a segment and a starting point for each identified segment.
In embodiments, the current segment information database is provided to, or acquired by, the slave server upon start-up, and in case resynchronization is required at the slave server or the master server, a more current segment information database is provided to, or acquired by, the slave server.
Advantageously, the slave server is initially synchronized with the master server in that the slave server is provided with the segment information database from which it initially can start creating segments. Thereby, segments of the slave server will advantageously be aligned with the segments of the master server. After the slave server is provided with the segment information database, it will proceed with creating segments as has been discussed in detail hereinabove in embodiments pertaining to the first and second aspect of the invention.
The master server generally performs, upon providing the segment information database to the slave server, one or more or all of the embodiments discussed above with reference to the first and second aspects of the invention. Further, the slave server performs, after having acquired the segment information database from the master server, one or more or all of the embodiments discussed above with reference to the first and second aspects of the invention, but generally not the embodiment of initially segmenting one of the at least two received content streams at a starting timestamp for creating a segment with which the other received content streams can be synchronized, since this is performed by the master server and is reflected in the segment information database provided to the slave server.
In a further aspect of the invention, computer programs are provided comprising computer-executable instructions for causing the servers to perform steps recited in the methods of the invention when the computer-executable instructions are executed on processing units included in the servers.
In yet a further aspect of the invention, computer program products are provided comprising computer readable media, the computer readable media having the computer programs embodied thereon.
Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to “a/an/the element, apparatus, component, means, step, etc.” are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.
The invention is now described, by way of example, with reference to the accompanying drawings, in which:
The invention will now be described more fully hereinafter with reference to the accompanying drawings, in which certain embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout the description.
In other implementations, a video stream may be provided at different resolutions, or an audio stream may be provided in different languages, which streams need to be segmented as in the example of ABS. In the following, the different profiles will be exemplified as content encoded at different bitrates.
The streaming server 30 receives the four different profiles and divides the corresponding content into data sets known as segments. The segments hence comprise the content at different bitrates. The client 40 can subsequently switch between the different segments, since corresponding segments of different profiles are equally-sized; if the client has capacity to stream a higher-quality segment, it can request to do so whereas if the capacity of the client 40 should decrease, it can switch to a lower-quality segment. Typically, the client reads a so called manifest file 50 stored at a streaming server 30, where the different segments and their associated quality/bitrate is identified.
The format of the content streams received at the streaming server 30 from the content source 20 could for instance include MPEG-2 (“Moving Picture Experts Group 2”) transport streams (TS) over separate multicast feeds, with synchronized timelines and reference frames. The format of the segmented streams output from the streaming server 30 to the client 40 may include Apple HLS (“Hypertext Transfer Protocol Live Streaming”), Microsoft Smooth, MPEG DASH (“Dynamic Adaptive Streaming over HTTP”), etc.
In ABS, the segments of different profiles must correspond to the same piece of the content to allow a client to switch between segments without experiencing a gap in the rendered content or a content overlap. This requires segmentation to be aligned across all profiles available to the streaming server 30 supplying the client 40 with content from the content source 20.
Now, as is illustrated in
For instance, if the client 40 renders Segment #1 of Profile #1, and then requests Segment #2 of Profile #2 by consulting the manifest file 50 of the streaming server 30, the streaming server 30 will provide created Segment #2 of Profile #2 accordingly, wherein in case the streaming server 30 would not have aligned the content streams received at its input side, there would be an overlap and the client 40 would render the very last part of Segment #1 before being able of rendering Segment #2 of Profile #2.
Correspondingly, without alignment of segments at the streaming server 30, if the client 40 would want to switch back to Profile #1 for created Segment #3 at BP#7 after having rendered Segment #2, there would be a “gap” in the content rendered at the client 40, and the client 40 would miss out on the very first part of Segment #3 of Profile #1.
Clearly, there is a need for alignment of the different content streams at the streaming server 30 for the client 40 to be able to render the different profiles in a satisfactory manner.
In this particular example, each segment extends over three BPs; for instance Segment #1 extends from BP#1 to BP#4. Typically, segment length is in practice between two and ten seconds. In the following, with reference to the illustration of
For each incoming profile, i.e. for each incoming content stream encoded at a particular bitrate, a current BP is identified in step S101 to determine if the stream can be segmented (if it has not already been segmented). As previously mentioned, each BP is associated with a corresponding timestamp (PTS). In this particular example, a segment information database 70 stored at the streaming server 30 initially holds one segment: Segment #1 at PTS#1. That is, segmenting of Profile 1 has been performed by the streaming server 30 to create Segment #1 starting at PTS#1.
Hence, in an embodiment, one of the received streams, in this example Profile #1, is initially segmented at a starting timestamp PTS#1 for creating a segment with which the other received content streams, in this case Profile #2, can be synchronized.
Now, in step S101, BP#1 at PTS#1 is identified for Profile #2 and thereafter, it is determined in step S102 whether a segment identifier is included in the segment information database 70, which identifier designates a segment having a starting point corresponding to the PTS of the identified BP; that is, whether a segment is identified in the segment information database 70 corresponding to PTS#1.
Hence, the streaming server 30 identifies BP#1 at PTS#1 for Profile #2 in step S101 and concludes in step S102 that PTS#1 is in the segment information database 70. The streaming server 30 will thus in step S103 create Segment #1 from the received content stream indicated by Profile #2, i.e. from the received stream a segment is created having the predetermined length extending over three BPs, and the identifier from the segment information database 70 is included with the created segment.
Advantageously, with the embodiment of the method of
The streaming server 30 then proceeds to BP#2 of Profile #1 in step S101, the timestamp PTS#2 of which is not in the manifest file 50 as is concluded in step S102. The streaming server thus proceeds to step S104, where it is determined whether PTS#2 of BP#2 indicates an allowable segment length. For PTS#2, that is not the case, and no segmenting is performed.
The same procedure is performed for BP#2 of Profile #2, with the same result.
Thereafter, the streaming server 30 identifies BP#3 at PTS#3 for Profile #1. Again, PTS#3 is not in the segment information database 70, and further does not indicate an allowable segment length. No segmenting is performed. The same procedure is performed for BP#3 of Profile #2, with the same result.
However, upon identifying BP#4 of Profile #1 at PTS#4 in step S101, and determining that the timestamp PTS#4 is not in the segment information database 70 in step S102, the streaming server 30 proceeds to step S104, where it is determined that PTS#4 of BP#4 indeed indicates an allowable segment length. As a consequence, the streaming server 30 updates the segment information database 70 and creates a new segment, i.e. Segment #2 of Profile #1, in step S105.
In an embodiment, the updating includes adding PTS#4 to the segment information database 70 along with an incremented segment identifier.
Thereafter, upon identifying BP#4 of Profile #2 at PTS#4 in step S101, and determining that the timestamp PTS#4 indeed is in the segment information database 70 in step S102, the streaming server 30 proceeds to step S103, where it creates a new segment, i.e. Segment #2 of Profile #2, and includes the identifier from the segment information database 70 with the created segment.
Advantageously, with the embodiment of the method of
First, it is determined in step S104a, whether a difference in time between the time stamp PTS of the identified current boundary point BP and a starting point of a closest previously identified segment exceeds a minimum allowable segment length. In the examples given hereinabove, a duration or length of a segment is defined to amount to three seconds. Thus, in case the streams comprise one BP every second, a minimum allowable segment length should exceed a time span covering three BPs, in this particular example a minimum allowable segment length should exceed two seconds.
Using the example of
Thereafter, the streaming server 30 identifies BP#3 at PTS#3 for Profile #1. Again, PTS#3 is not in the segment information database 70, and further does not indicate an allowable segment length, since PTS#3-PTS#1 <minimum segment length; no segmenting is performed.
However, upon identifying BP#4 of Profile #1 at PTS#4 in step S101, and determining that the timestamp PTS#4 is not in the segment information database 70 in step S102, the streaming server 30 proceeds to step S104a, where it is determined that PTS#4 of BP#4 indeed indicates an allowable segment length, since PTS#4-PTS#1>minimum segment length.
The streaming server 30 optionally proceeds to step S104c, where it determines whether a difference in time between the PTS of the identified current BP and a starting point of a closest previously identified segment is less than a maximum allowable segment length.
In this example, the duration or length of a segment is defined to amount to three seconds. Thus, with the example given with reference to
In step S104c, it is determined that PTS#4-PTS#1<maximum segment length, and the streaming server 30 proceeds to step S105 for updating segment information database 70 and creating a new segment, i.e. Segment #2 of Profile #1.
For the sake of discussion, if the currently identified BP would have been BP#5, without BP#4 having been identified previously for Profile #1, the streaming server 30 would have determined in step S104c that BP#5-BP#1 >maximum segment length, and proceeded to step S104d where it would have concluded that synchronization with Profile #1 is lost.
In such a case, in step S104d, the streaming server 30 advantageously re-synchronizes with Profile #1, for instance at BP#7 where Segment #3 is created at a new starting timestamp PTS#7 for creating a segment with which the other received content streams, in this case Profile #2, can be resynchronized.
For redundancy, the same content streams are typically sent to multiple streaming servers 30, 60. In such a scenario, it is then crucial that the segmentation and identification of segments are aligned across the multiple streaming servers, to allow clients to perform failover to another server during a streaming session.
Hence, not only should the segments of each content stream Profile #1, Profile #2 be aligned internally with in each server 30, 60, but further the segments output from the first server 30 should be aligned with those output from the second server 60 in order to provide for the client 40 to switch between segments of the first server 30 and the second server 60.
As is illustrated in
Thus, with reference to
The slave server 60 is hence initially synchronized with the master server 30 in that the slave server 60 is provided with the segment information database 70 from which it initially can start creating segments. Thereby, segments of the slave server 60 will advantageously be aligned with the segments of the master server 30, and the client 40 is able to render segments of the different profiles in a satisfactory manner. After the slave server 60 is provided with the segment information database 70, it will proceed with creating segments as has been discussed in detail hereinabove with reference to
Alternatively, with reference to
Hence, upon start-up of the slave server 60, the current segment information database 70 is transferred from the master server 30 to the slave server 60, either by the slave server 60 requesting the database (S100b), or the master server 30 being configured to provide the database 70 to the slave server (S100a) upon start-up of the slave server 60.
Now, in step S101, BP#1 at PTS#1 is identified for Profile #2 and thereafter, it is determined in step S102 whether a segment identifier is included in the segment information database 70, which identifier designates a segment having a starting point corresponding to the PTS of the identified BP; that is, whether a segment is identified in the segment information database 70 corresponding to PTS#1.
Hence, the slave server 60 identifies BP#1 at PTS#1 for Profile #2 in step S101 and concludes in step S102 that PTS#1 is in the segment information database 70. The slave server 60 will thus in step S103 create Segment #1 from the received content stream indicated by Profile #2, i.e. from the received stream a segment is created having the predetermined length extending over three BPs, and the identifier from the segment information database 70 is included with the created segment.
The slave server 60 proceeds to BP#2 of Profile #1 in step S101, the timestamp PTS#2 of which is not in the segment information database 70 as is concluded in step S102. The slave server 60 thus proceeds to step S104a, where it is determined whether the difference between PTS#2 and a starting point of a closest previously identified segment, i.e. PTS#1, exceeds the minimum allowable segment length, which is not the case, and PTS#2 of the identified current boundary point B#2 is precluded as a possible segmenting point in step S104b.
Thereafter, the streaming server 30 identifies BP#3 at PTS#3 for Profile #1 in step S101. Again, PTS#3 is not in the segment information database 70 as concluded in step S102, and further does not indicate an allowable segment length in step S104a, since PTS#3-PTS#1<minimum segment length; no segmenting is performed, step S104b.
However, upon identifying BP#4 of Profile #1 at PTS#4 in step S101, and determining that the timestamp PTS#4 is not in the segment information database 70 in step S102, the slave server 60 proceeds to step S104a, where it is determined that PTS#4 of BP#4 indeed indicates an allowable segment length, since PTS#4-PTS#1>minimum segment length.
The slave server 60 optionally proceeds to step S104c, where it determines whether a difference in time between the PTS of the identified current BP and a starting point of a closest previously identified segment is less than a maximum allowable segment length.
In this example, the duration or length of a segment is defined to amount to three seconds, with BPs occurring each second. Thus, a maximum allowable segment length should be less than four seconds.
In step S104c, it is determined that PTS#4-PTS#1<maximum segment length, and the slave server 60 proceeds to step S105 for updating segment information database 70 and creating a new segment, i.e. Segment #2 of Profile #1.
Hence, by acquiring the current segment information database from the master server 30, the created segments of the slave server 60 will advantageously be aligned with those of the master server 30.
However, if the currently identified BP would have been BP#5, without BP#4 having been identified previously for Profile #1, the slave server 60 would have determined in step S104c that BP#5-BP#1>maximum segment length, and proceeded to step S104d where it would have concluded that synchronization with Profile #1 is lost.
In such a case, in step S104d, the slave server 60 advantageously acquires a more current segmentation database 70 from the master server 30, and the slave server 60 is advantageously resynchronized with the master server 30. The process of the slave server 60 thereafter commences again at step S101.
With reference to
The invention has mainly been described above with reference to a few embodiments. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the invention, as defined by the appended patent claims.
Number | Date | Country | Kind |
---|---|---|---|
1551456-5 | Nov 2015 | SE | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/SE2016/051003 | 10/17/2016 | WO | 00 |