Patent Application
20240364861
The present invention is concerned with coding pictures, and in particular a consecutive sequence of pictures. Some embodiments may exploit scalable coding and/or the so-called Gradual Decoder Refresh-GDR-coding scheme for coding the pictures. Some embodiments may suggest Scalable Coding and Gradual Decoder Refresh improvements.
In nowadays video coding, some scenarios require very low latency transmission. Whenever low-latency end2end delays are required, bitrate changes from picture to picture are undesirable. Typically, video is encoded in such a way that the sizes of the encoded pictures vary not only due to complexity characteristics of the content but also by the prediction structure used. More concretely, typically, videos are encoded using a prediction structure that is based on some pictures being encoded as Intra slices (not dependent on other pictures) and others being encoded as B or P slices (using other pictures as references). Obviously, pictures being encoded without prediction to other pictures lead to bigger sizes than when temporal correlation is used and pictures are encoded as B or P slices.
There are some techniques that drift a bit of this typical encoding structure where each of the picture is encoded with some mixture of blocks using only intra prediction and some blocks using inter-prediction. In such a case, there is not a picture that is encoded only using intra prediction and therefore (if the ratio of intra predicted blocks and inter predicted blocks is kept similar across all pictures) the sizes of all pictures is kept similar.
This is ideal to achieve a lower end2end delay as the size of the biggest picture compared to the other coding structure approach is smaller and therefore the time to transmit it becomes smaller as well.
Typically, such structures are identified as Gradual Decoding Refresh (GDR) as they differ from typical coding structures in the fact that, in order to achieve a clean picture, several pictures need to be decoded and the video areas are gradually decoded and refreshed until the content can be properly shown; while in typical coding structures, only a particular form of an Intra frame (Random Access Point-RAP) is required to be present and the content can be instantaneously shown without having to decode more access units.
In accessing a frame or picture in a picture sequence, there may be a trade-off between the so-called tune-in time and the coding efficiency. Mechanisms that allow reducing the tune-in time (either on average or in worst case) while not harming the coding efficiency are desirable. Furthermore, a picture may be subdivided in picture regions (e.g. tiles) which may be refreshed over time in a so-called Refresh Period RP. However, the identification of what regions are clean (refreshed) and what regions are not refreshed is not clear, and thus comes with some penalties, such as, intra prediction cannot be easily restricted between not-yet clean (dirty) regions and clean regions.
Thus, it is an object of the present invention to improve existing GDR encoders, decoders and data streams.
A first aspect concerns a video data stream comprising a sequence of pictures comprising at least one Gradual Decoder Refresh-GDR-coded picture and one or more subsequent pictures in a refresh period (RP). The video data stream further comprises a parameter set (SPS) defining a plurality of picture configurations, which subdivide a picture area (e.g. entire frame) into a first sub-area and a second sub-area among which one corresponds to a refreshed sub-area (e.g. a set of picture regions) comprising one or more refreshed picture regions (e.g. tiles) and the other one corresponds to an un-refreshed sub-area comprising one or more yet un-refreshed picture regions. The video data stream further comprises, for each picture within the refresh period, a picture configuration identifier (reg_conf_idx) for identifying a corresponding one picture configuration out of the plurality of picture configurations.
Furthermore, it is suggested to provide a decoder for decoding from a data stream at least one picture out of a sequence of pictures comprising at least one Gradual Decoder Refresh—GDR—coded picture and one or more subsequent pictures in a refresh period (RP). The decoder is configured to read from the data stream a parameter set (SPS) defining a plurality of picture configurations, which subdivide a picture area (e.g. entire frame) into a first sub-area and a second sub-area among which one corresponds to a refreshed sub-area (e.g. a set of picture regions) comprising one or more refreshed picture regions (e.g. tiles) and the other one corresponds to an un-refreshed sub-area comprising one or more yet un-refreshed picture regions. The decoder is further configured to read from the data stream, for each picture within the refresh period, a picture configuration identifier (reg_conf_idx) for identifying a corresponding one picture configuration out of the plurality of picture configurations for decoding the at least one picture.
Furthermore, it is suggested to provide an encoder for encoding into a data stream at least one picture out of a sequence of pictures comprising at least one Gradual Decoder Refresh—GDR—coded picture and one or more subsequent pictures in a refresh period (RP). The encoder is configured to write into the data stream a parameter set (SPS) defining a plurality of picture configurations, which subdivide a picture area (e.g. entire frame) into a first sub-area and a second sub-area among which one corresponds to a refreshed sub-area (e.g. a set of picture regions) comprising one or more refreshed picture regions (e.g. tiles) and the other one corresponds to an un-refreshed sub-area comprising one or more yet un-refreshed picture regions. The encoder is further configured to set in the data stream, for each picture within the refresh period, a picture configuration identifier (reg_conf_idx) for identifying a corresponding one picture configuration out of the plurality of picture configurations.
A second aspect concerns a video data stream comprising a sequence of pictures comprising at least one Gradual Decoder Refresh-GDR-coded picture and one or more subsequent pictures in a refresh period, wherein each picture of the sequence of pictures is sequentially coded into the video data stream in units of blocks (e.g. CTUs) into which the respective picture is subdivided. The video data stream comprises an implicit signaling, wherein a refreshed sub-area of a respective picture is implicitly signaled in the video data stream based on a block coding order. Additionally or alternatively, the video data stream comprises, for each block, a syntax element (e.g. a flag) indicating whether
Furthermore, it is suggested to provide a decoder for decoding from a data stream at least one picture out of a sequence of pictures comprising at least one Gradual Decoder Refresh—GDR—coded picture and one or more subsequent pictures in a refresh period (RP), wherein each picture of the sequence of pictures is sequentially decoded from the video data stream in units of blocks (e.g. CTUs) into which the respective picture is subdivided. The decoder is configured to implicitly derive from the data stream a refreshed sub-area of the at least one picture based on a block coding order. Additionally or alternatively, the decoder is configured to read from the data stream, for each block, a syntax element (e.g. a flag) indicating whether
Furthermore, it is suggested to provide an encoder for encoding into a data stream at least one picture out of a sequence of pictures comprising at least one Gradual Decoder Refresh—GDR—coded picture and one or more subsequent pictures in a refresh period (RP), wherein each picture of the sequence of pictures is sequentially encoded into the video data stream in units of blocks (e.g. CTUs) into which the respective picture is subdivided. The encoder is configured to write into the data stream, for each block, a syntax element (e.g. a flag) indicating whether
A third aspect concerns a multi layered scalable video data stream comprising a first sequence of pictures in a first layer (e.g. base layer) and a second sequence of pictures in a second layer (e.g. an enhancement layer), wherein the second sequence of pictures in the second layer comprises at least one Gradual Decoder Refresh-GDR-picture as a start picture and one or more subsequent pictures in a refresh period. The multi layered scalable video data stream comprises a signalization carrying information about a possibility that a yet un-refreshed sub-area of the GDR picture of the second layer is to be inter-layer predicted from samples of the first layer. Additionally, the multi layered scalable video data stream comprises information:
Furthermore, it is suggested to provide a decoder for decoding at least one picture from a multi layered scalable video data stream comprising a first sequence of pictures in a first layer (e.g. base layer) and a second sequence of pictures in a second layer (e.g. an enhancement layer), wherein the second sequence of pictures in the second layer comprises at least one Gradual Decoder Refresh-GDR-picture as a start picture and one or more subsequent pictures in a refresh period. The decoder is configured to read from the multi layered scalable video data stream a signalization carrying information about a possibility that a yet un-refreshed sub-area of the GDR picture of the second layer is to be inter-layer predicted from samples of the first layer. The decoder is further configured to, responsive to the signalization:
Furthermore, it is suggested to provide an encoder for encoding at least one picture into a multi layered scalable video data stream comprising a first sequence of pictures in a first layer (e.g. base layer) and a second sequence of pictures in a second layer (e.g. an enhancement layer), wherein the second sequence of pictures in the second layer comprises at least one Gradual Decoder Refresh-GDR-picture as a start picture and one or more subsequent pictures in a refresh period. The encoder is configured to write into the multi layered scalable video data stream a signalization carrying information about a possibility that a yet un-refreshed sub-area of the GDR picture of the second layer is to be inter-layer predicted from samples of the first layer, and information:
A fourth aspect concerns a multi layered scalable video data stream comprising a first sequence of pictures in a first layer (e.g. base layer) and a second sequence of pictures in a second layer (e.g. an enhancement layer), each of the first and second layers comprising a plurality of temporal sublayers. The scalable video data stream further comprises a signalization (e.g. vps_sub_layer_independent_flag [i] [j]) indicating which temporal sublayers of the second layer (e.g. enhancement layer) are coded by inter-layer prediction.
Furthermore, it is suggested to provide a decoder for decoding at least one picture from a multi layered scalable video data stream comprising a first sequence of pictures in a first layer (e.g. base layer) and a second sequence of pictures in a second layer (e.g. an enhancement layer), each of the first and second layers comprising a plurality of temporal sublayers. The decoder is configured to decode one or more of the temporal sublayers by using inter-layer prediction based on a signalization derived from the scalable video data stream, said signalization (e.g. vps_sub_layer_independent_flag[i][j]) indicating which temporal sublayers of the second layer (e.g. enhancement layer) are to be coded by inter-layer prediction.
Furthermore, it is suggested to provide an encoder for encoding at least one picture into a multi layered scalable video data stream comprising a first sequence of pictures in a first layer (e.g. base layer) and a second sequence of pictures in a second layer (e.g. an enhancement layer), each of the first and second layers comprising a plurality of temporal sublayers. The encoder is configured to encode one or more of the temporal sublayers by using inter-layer prediction and to write a signalization into the scalable video data stream, said signalization (e.g. vps_sub_layer_independent_flag [i] [j]) indicating which temporal sublayers of the second layer (e.g. enhancement layer) are coded by inter-layer prediction.
A fifth aspect concerns a video data stream comprising at least one picture being subdivided into tiles, and a tile-reordering flag, wherein
Furthermore, it is suggested to provide a decoder configured to decode a picture from a data stream, wherein:
Furthermore, it is suggested to provide an encoder configured to encode a picture into a data stream, wherein:
In the following, embodiments of the present disclosure are described in more detail with reference to the figures, in which
Equal or equivalent elements or elements with equal or equivalent functionality are denoted in the following description by equal or equivalent reference numerals.
Method steps which are depicted by means of a block diagram and which are described with reference to said block diagram may also be executed in an order different from the depicted and/or described order. Furthermore, method steps concerning a particular feature of a device may be replaceable with said feature of said device, and the other way around.
Furthermore, in this disclosure the terms frame and picture may be used interchangeably.
A picture comprising an Intra-Coded picture region 102a may also be referred to as a GDR-picture 103. In this example, every second picture may be a GDR-picture 103. Accordingly, the GDR-delta is two in this example. A picture region that has been Intra-Coded (i.e. an Intra-Coded picture region 102a) may also be referred to as a refreshed picture region or a clean picture region, respectively. Picture regions that have not yet been coded after an access may be referred to as non-refreshed picture regions or dirty picture regions, respectively.
The refresh period (RP) is the time interval that needs to be waited until all picture regions 102 are refreshed and a clean picture can be shown. There are different forms of configuring such a bitstream. In
Furthermore, in this example, a GDR picture 103 may be present every second picture, i.e. a picture at which a decoder can start accessing the bitstream and decode a full picture after a full RP. This can be achieved by encoding all picture regions 102 independently of each other over the time, e.g. called MCTS in HEVC and spreading the intra coded blocks 102a in time with a distance of two frames among picture regions 102 in the example above.
Variant B: Constrained Inter Tiles (c.f. Attached
In the non-limiting example shown in
In summary, for all of the above discussed non-limiting example variants of GDR, the RP may be 17 and the GDR periodicity may be Δ=2 or Δ=18 as shown in the table below.
A further important aspect to evaluate a GDR technology is the tune-in time required to show a picture, which consists of the RP plus the time that needs to be waited until a GDR picture 103 is found. The table above shows the tune-in-time in average and worst-case.
In order to solve the issue of regions changing from picture to picture (i.e. a change from not-yet refreshed picture regions to a refreshed picture region), two inventive approaches can be envisioned to avoid the burden of having to send an updated PPS with every picture:
In the first inventive approach, several configurations of the picture regions, e.g. in form of tiles, may be signaled within the SPS (Sequence Parameter Set) or PPS (Picture Parameter Set) and slice headers may point to an index to indicate which is the tile configuration that is in use for a given AU (Access Unit). Thus, a dynamic signaling of refreshed and yet un-refreshed picture regions may be provided with this inventive approach.
In this disclosure, a picture area may comprise an entire frame 101 or picture 101. A picture area may be divided into picture sub-areas. As exemplarily shown in the fourth frame in
According to an embodiment of the first aspect of the present invention, a video data stream may be provided, the video data stream comprising a sequence 100 of pictures 1011, 1012, . . . , 101n comprising at least one Gradual Decoder Refresh-GDR-coded picture 103 and one or more subsequent pictures in a refresh period RP. The video data stream further comprises a parameter set (e.g. SPS or PPS) defining a plurality of picture configurations, which subdivide a picture area 101 (e.g. an entire frame) into a first sub-area 101r (e.g. first one or more picture regions 102 comprising tiles, rows, columns, etc.) and a second sub-area 101u (e.g. second one or more picture regions 102 comprising tiles, rows, columns, etc.) among which one sub-area corresponds to a refreshed sub-area 101r comprising one or more (i.e. a set of) refreshed picture regions (e.g. tiles) and the other sub-area 101u corresponds to an un-refreshed sub-area comprising one or more (=a set of) yet un-refreshed picture regions. According to the inventive principle, the video data stream comprises for each picture 1011, 1012, . . . , 101n within the refresh period RP a picture configuration identifier (e.g. region_configuration_idx) for identifying a corresponding one picture configuration out of the plurality of picture configurations.
According to a further embodiment, it is suggested to provide a corresponding decoder for decoding from a data stream at least one picture out of a sequence 100 of pictures 1011, 1012, . . . , 101n comprising at least one Gradual Decoder Refresh-GDR-coded picture 103 and one or more subsequent pictures 1012, . . . , 101n in a refresh period-RP, wherein the decoder is configured to read from the data stream a parameter set (e.g. PPS or SPS) defining a plurality of picture configurations, which subdivide a picture area (e.g. an entire frame) 101 into a first sub-area 101r and a second sub-area 101u among which one corresponds to a refreshed sub-area (e.g. a set of refreshed picture regions) 101r comprising one or more refreshed picture regions (e.g. refreshed tiles) 102r and the other one corresponds to an un-refreshed sub-area 101u comprising one or more yet un-refreshed picture regions 102u. The decoder may further be configured to read from the data stream, for each picture 1011, 1012, . . . , 101n within the refresh period—RP, a picture configuration identifier (region_configuration_idx) for identifying a corresponding one picture configuration out of the plurality of picture configurations for decoding the at least one picture 1011, 1012, . . . , 101n.
According to a further embodiment, it is suggested to provide a corresponding encoder for encoding into a data stream at least one picture out of a sequence 100 of pictures 1011, 1012, . . . , 101n comprising at least one Gradual Decoder Refresh-GDR-coded picture 103 and one or more subsequent pictures 1012, . . . , 101n in a refresh period-RP, wherein the encoder is configured to write into the data stream a parameter set defining a plurality of picture configurations, which subdivide a picture area 101 into a first sub-area 101r and a second sub-area 101u among which one corresponds to a refreshed sub-area 101r comprising one or more refreshed picture regions 102r and the other one corresponds to an un-refreshed sub-area 101u comprising one or more yet un-refreshed picture regions 102u. The encoder is further configured to set in the data stream, for each picture 1011, 1012, . . . , 101n within the refresh period-RP, a picture configuration identifier for identifying a corresponding one picture configuration out of the plurality of picture configurations for decoding the at least one picture 1011, 1012, . . . , 101n.
According to a further embodiment, it is suggested to provide a corresponding method for decoding from a data stream at least one picture out of a sequence 100 of pictures 1011, 1012, . . . , 101n comprising at least one Gradual Decoder Refresh-GDR-coded picture 103 and one or more subsequent pictures 1012, . . . , 101n in a refresh period-RP, the method comprising steps of reading from the data stream a parameter set defining a plurality of picture configurations, which subdivide a picture area 101 into a first sub-area 101r and a second sub-area 101u among which one corresponds to a refreshed sub-area 101r comprising one or more refreshed picture regions 102r and the other one corresponds to an un-refreshed sub-area 101u comprising one or more yet un-refreshed picture regions 102u. The method further comprises a step of reading from the data stream, for each picture 1011, 1012, . . . , 101n within the refresh period-RP, a picture configuration identifier for identifying a corresponding one picture configuration out of the plurality of picture configurations for decoding the at least one picture 1011, 1012, . . . , 101n.
According to a further embodiment, it is suggested to provide a corresponding method for encoding into a data stream at least one picture out of a sequence 100 of pictures 1011, 1012, . . . , 101n comprising at least one Gradual Decoder Refresh-GDR-coded picture 103 and one or more subsequent pictures 1012, . . . , 101n in a refresh period-RP, the method comprising steps of writing into the data stream a parameter set defining a plurality of picture configurations, which subdivide a picture area 101 into a first sub-area 101r and a second sub-area 101u among which one corresponds to a refreshed sub-area 101r comprising one or more refreshed picture regions 102r and the other one corresponds to an un-refreshed sub-area 101u comprising one or more yet un-refreshed picture regions 102u. The method further comprises a step of setting in the data stream, for each picture 1011, 1012, . . . , 101n within the refresh period-RP, a picture configuration identifier for identifying a corresponding one picture configuration out of the plurality of picture configurations.
As mentioned above, the sequence 100 of pictures 1011, 1012, . . . , 101n may be coded in different ways. For instance, the sequence 100 of pictures 1011, 1012, . . . , 101n may be coded in a manner so that intra prediction does not cross a boundary between the first and second sub-areas 101r, 101u. Additionally or alternatively, the sequence 100 of pictures 1011, 1012, . . . , 101n may be coded in a manner so that temporal prediction of the refreshed sub-area 101r does not reference the yet un-refreshed sub-area 101u. Additionally or alternatively, the sequence 100 of pictures 1011, 1012, . . . , 101n may be coded in a manner so that context model derivation does not cross a boundary between the first and second sub-areas 101r, 101u.
According to an advantageous embodiment, the corresponding one picture configuration indicates the refreshed picture regions 102r and the yet un-refreshed picture regions 102u contained in a currently coded picture 1011, 1012, . . . , 101n of the sequence 100 of pictures.
According to a further advantageous embodiment, each picture configuration out of the plurality of picture configurations may comprise a set of region indices for signaling which picture regions 102 are refreshed picture regions 102r and which picture regions 102 are un-refreshed picture regions 102u. This provides for an explicit signaling of refreshed and yet un-refreshed picture regions 102r, 102u.
For example, as shown and previously discussed with reference to
According to a further embodiment, as exemplarily discussed with reference to
According to a further embodiment, as exemplarily discussed with reference to
A picture region may also be represented by a coding block, e.g. by a CTU (Coding Tree Unit).
According to a further embodiment, the pictures 1011, 1012, . . . , 101n contained in the sequence 100 of pictures may be subdivided into rows of coding blocks (e.g. CTUs), wherein each picture configuration out of the plurality of picture configurations may comprise at least one row coding block index for signaling which rows of coding blocks are refreshed rows 102r of coding blocks and/or which rows of coding blocks are un-refreshed rows 102u of coding blocks. Accordingly, the region configuration may contain a CTU row index.
According to a further embodiment, the pictures 1011, 1012, . . . , 101n contained in the sequence 100 of pictures may be subdivided into columns of coding blocks (e.g. CTUs), wherein each picture configuration out of the plurality of picture configurations may comprise at least one column coding block index for signaling which columns of coding blocks are refreshed columns 102r of coding blocks and/or which columns of coding blocks are un-refreshed columns 102u of coding blocks. Accordingly, the region configuration may contain a CTU column index.
According to a further embodiment, the pictures 1011, 1012, . . . , 101n contained in the sequence 100 of pictures may be subdivided into diagonals of coding blocks (e.g. CTUs), and wherein each picture configuration out of the plurality of picture configurations may comprise at least one diagonal coding block index for signaling which diagonals of coding blocks are refreshed diagonals 102r of coding blocks and/or which diagonals of coding blocks are un-refreshed diagonals 102u of coding blocks. Accordingly, the region configuration may contain one or more indices of a CTU diagonal.
A picture region may also be represented by samples.
According to a further embodiment, the pictures 1011, 1012, . . . , 101n contained in the sequence 100 of pictures may be subdivided into rows of samples, wherein each picture configuration out of the plurality of picture configurations may comprise at least one sample row index for signaling which rows of samples are refreshed rows 102r of samples and/or which rows of samples are un-refreshed rows 102u of samples. Accordingly, the region configuration may contain one or more sample row indexes.
According to a further embodiment, the pictures 1011, 1012, . . . , 101n contained in the sequence 100 of pictures may be subdivided into columns of samples, wherein each picture configuration out of the plurality of picture configurations may comprise at least one sample column index for signaling which columns of samples are refreshed columns 102r of samples and/or which columns of samples are un-refreshed columns 102u of samples. Accordingly, the region configuration may contain one or more sample column indexes.
According to a yet further embodiment, the corresponding one picture configuration may be signaled in a slice header and/or in an Access Unit Delimiter of the video data stream.
Then the slice header would indicate which configuration is used:
Additionally or alternatively, the information about the used region configuration is included into the Access Unit Delimiter (AUD).
According to a further embodiment, it is suggested to provide a video data stream comprising a sequence 100 of pictures 1011, 1012, . . . , 101n comprising at least one Gradual Decoder Refresh-GDR-coded picture 103 and one or more subsequent pictures 1012, . . . , 101n in a refresh period-RP. Each picture of the sequence 100 of pictures 1011, 1012, . . . , 101n may be sequentially coded into the video data stream in units of blocks 102 (e.g. CTUs) into which the respective picture is subdivided. The video data stream may comprise an implicit signaling, wherein a refreshed sub-area 102r of a respective picture 1011, 1012, . . . , 101n is implicitly signaled in the video data stream based on a block coding order.
Also, a respective decoder is suggested, i.e. a decoder for decoding from a data stream at least one picture out of a sequence 100 of pictures 1011, 1012, . . . , 101n comprising at least one Gradual Decoder Refresh-GDR-coded picture 103 and one or more subsequent pictures in a refresh period RP. Each picture of the sequence 100 of pictures 1011, 1012, . . . , 101n may be sequentially decoded from the video data stream in units of blocks 102 (e.g. CTUs) into which the respective picture is subdivided. The decoder may be configured to implicitly derive from the data stream a refreshed sub-area 102r of the at least one picture based on a block coding order.
Also, a respective encoder is suggested, i.e. an encoder for encoding into a data stream at least one picture out of a sequence 100 of pictures 1011, 1012, . . . , 101n comprising at least one Gradual Decoder Refresh-GDR-coded picture 103 and one or more subsequent pictures in a refresh period RP. Each picture of the sequence 100 of pictures 1011, 1012, . . . , 101n may be sequentially encoded into the video data stream in units of blocks 102 (e.g. CTUs) into which the respective picture is subdivided. The encoder may be configured to implicitly derive from the data stream a refreshed sub-area 101r of the at least one picture based on a block coding order.
Thus, in this approach a CTU based signaling of the region boundary may be used. Each CTU (picture region) 102 may contain a flag (which can be CABAC-coded) indicating whether it is the last CTU 102 of the GDR region 103 or not. This signaling affects for instance the availability of samples for intra prediction and/or CABAC reset. The benefit of such an approach is that it is more flexible not being limited to a fixed grid defined in a parameter set.
Thus, according to an embodiment, the syntax element is for indicating a boundary between a refreshed sub-area 101r and a yet un-refreshed sub-area 101u of a picture 101 out of the sequence 100 of pictures 1011, 1012, . . . , 101n and/or for indicating which sub-area is a refreshed sub-area 101r and which sub-area is a yet un-refreshed sub-area 101u.
With each CTU 102 indicating whether it is the last CTU in the GDR region 103, it is beneficial to identify whether the last CTU 102 in a region means the last CTU 102 in terms of rows or columns. Such an indication may be done in a parameter set, e.g. in SPS.
For example, the flag region_horizontal_flag equal to 1 may indicate that the last CTU flag in CTU indicates a horizontal split. Otherwise, a vertical split.
Thus, according to a further embodiment, a video data stream, an encoder and a decoder are suggested, wherein
According to a further embodiment, the further syntax element may indicate whether the last block derives from a horizontal split (e.g. region_horizontal_flag=1) or from a vertical split of a coding split tree according to which the respective picture 101 is subdivided into blocks 102.
In a further embodiment it may be indicated whether the region is an intra-prediction break, i.e. neighbors of another region are not available for prediction, or CABAC, etc. Thus, the video data stream may comprise an intra-prediction break indication for indicating that neighboring blocks 102 of a neighboring picture region 102u are not available for prediction, e.g. if said neighboring picture region 102u is contained in a yet un-refreshed sub-area 101u.
In both cases defined above the grid of the regions used may be aligned to the CTU sizes. In other words, the refreshed sub-area 101r may comprise one or more refreshed picture regions 102r which are arranged in a grid that is aligned with the size of the blocks 102 into which the respective picture 101 is subdivided.
In the embodiments described so far, it is not necessarily known which of the regions is a refreshed (clean) and not refreshed (dirty) region. In this case, all regions are considered to be “independent” from each other in all or some of the following aspects:
Alternatively, the signaling implicitly indicates that the left-most region 101r is a clean region and the availability of intra blocks is constrained for this region—i.e. the blocks 102 in the left-most region cannot use blocks of another (non-left-most) region for all or some of the following aspects:
Thus, according to an embodiment, the implicit signaling may signal that a first block 102 at a predetermined position in the block coding order (e.g. a first CTU in upper left corner) is part of the refreshed-sub area 101r.
As an alternative to the above described implicit derivation of the refreshed and yet un-refreshed regions 102r, 102u, some embodiments of the present invention may provide for an explicit signaling, wherein video data stream, a respective encoder and a respective decoder are suggested, wherein the video data stream may comprise, for each block 102, a syntax element indicating whether
In other words, as an alternative to the above described implicit derivation of the refreshed and yet un-refreshed regions 102r, 102u, it is suggested to explicitly indicate which region is a clean region 102r and which not, as discussed in the following embodiments.
In an embodiment, in addition to indicating the end of the GDR region 103, also the start of the GDR region 103 may be indicated at CTU level, e.g. by using a flag (CABAC-coded)
This would be helpful for an MCTS-like refresh-approach (c.f.
In another embodiment the CTU based region start and/or end flags may be signaled only in the first CTU column of a tile 102, if horizontal region splits are enabled, and in the first CTU row of a tile 102, if vertical region splits are enabled.
Thus, according to an embodiment, a picture region of the respective picture 101 may be vertically subdivided into one or more slices 102, wherein the syntax element (e.g. flag: last_ctu_of_gdr_region//flag: first_ctu_of_gdr_region) is signaled for each slice 102.
According to a further embodiment, a picture region of the respective picture 101 may be horizontally subdivided into one or more rows of blocks 102, wherein the syntax element (e.g. flag: last_ctu_of_gdr_region//flag: first_ctu_of_gdr_region) is signaled
In another embodiment a CTU based (CABAC-coded) flag may be signaled, indicating whether the CTU 102 is part of the GDR refresh region 103 or not.
In another embodiment the CTU start and/or end indexes of the GDR refresh region 103 may be signaled in the slice header.
One of the benefits of the above described embodiments is that picture regions may be decoupled from the usage of tiles 102 and thereby the scan order may not be affected. In most of the applications which use GDR, low delay transmission is desired. In order to achieve low delay transmission, all packets sent should have the same size and not only all AUs. Typically, in those low delay scenarios each AU may be split into multiple packets and in order to achieve that all packets have the same size (or very similar), each packet should have the same amount of blocks 102r that are refreshed (belong to the clean area 101r) and of blocks 102u that are not refreshed (belong to the dirty area 101u).
If tiles were used for that purpose, the tile scan order would be in use and therefore the packets 501a, 501b, . . . , 501n could not have the same amount of blocks 102r that are refreshed (belong to the clean area 101r) and of blocks 102u that are not refreshed (belong to the dirty area 101u).
In another embodiment tiles are used but a syntax element is added to the parameter set that enforces to follow raster scan and not tile scan. E.g., sps_enfoce_raster_scan_flag. In that case, raster scan would be used and not byte alignment would happen within the bitstream for CTUs starting a new tile.
Thus, according to an embodiment, a video data stream is suggested comprising at least one picture 101 being subdivided into tiles 102, and a tile-reordering flag (e.g. sps_enforce_raster_scan_flag), wherein
A further embodiment suggests a corresponding decoder that may be configured to decode a picture 101 from a data stream, wherein:
As mentioned above with reference to
A further embodiment suggests a corresponding encoder configured to encode a picture 101 into a data stream, wherein:
In case that GDR is done for a scalable bitstream, it could be possible to have a RP as discussed herein where the highest quality is achieved, while at the same time a low-quality RP (LQRP) that is smaller could be achieved, where a not-yet refreshed region 101u at the highest quality may be substituted with samples of the low quality content of a lower layer.
In one embodiment, the decoding process of the EL 602 would manage the status of the defined GDR regions (refreshed since GDR or not) and indicate for each region whether its initialized per layer or not. If a region is not initialized, the resampling process of a reference layer 601 for that region would be carried out and sample values would be substituted. Thereby, when decoding starts at the access unit containing the EL GDR picture 103, higher layer pictures can instantly be presented to the user, gradually being updated to the EL quality over the course of a RP.
However, constraints in the bitstreams are necessary for the above procedure to function.
Thus, according to an embodiment, a multi layered scalable video data stream 600 is suggested comprising a first sequence 200 of pictures 2011, 2012, . . . , 201n in a first layer (e.g. base layer) 601 and a second sequence 100 of pictures 1011, 1012, . . . , 101n in a second layer (e.g. an enhancement layer) 602. The second sequence 100 of pictures 1011, 1012, . . . , 101n in the second layer 602 may comprise at least one Gradual Decoder Refresh-GDR-picture 103 as a start picture and one or more subsequent pictures in a refresh period—RP, wherein the multi layered scalable video data stream 600 may comprise a signalization carrying information about a possibility that a yet un-refreshed sub-area 101u of the GDR picture 103 of the second layer 602 is to be inter-layer predicted from samples 202r of the first layer 601. The signalization may further carry information:
In one embodiment, TMVP (Temporal Motion Vector Prediction) or sub-block TMVP (i.e. syntax based prediction of motion vectors) may be disabled for the non-refreshed regions 101u in GDR pictures 103 so that when samples 102u are substituted by upsampled BL samples 202r, the following EL pictures 1011, . . . , 101n can use the substituted samples 202r for prediction, which significantly reduces encoder/decoder drift compared to using wrong motion vectors as would occur without the constraint.
Accordingly, a basic principle of this aspect suggests to provide the multi layered scalable video data stream 600, wherein said inter-layer prediction from samples 202r of the first layer 601 may comprise substituting one or more samples 102u of the yet un-refreshed sub-area 101u of the GDR picture 103 by an upsampled version of refreshed samples 202r of the first layer 601.
A further embodiment suggests that all samples 102u of the entire yet un-refreshed sub-area 101u of the GDR picture 103 may be substituted by an upsampled version of refreshed samples 202r of the first layer 601 so that pictures 1011, 1012, . . . , 101n from the second sequence 100 of coded pictures in the second layer 602 may be instantly presentable to a user.
A further embodiment suggests that yet un-refreshed sub-areas 101u of the one or more subsequent pictures 1012, . . . , 101n of the second layer 100 may be refreshed by intra-layer prediction (e.g. inside the second layer 602) using the upsampled substitute samples 202r from the first layer 601 which are gradually updated to refreshed samples 102r of the second layer 602.
In another embodiment, combined motion vector candidates in the merge list that are influenced by TMVP or sub-block TMVP candidates are forbidden so that when samples 101u are substituted by upsampled BL samples 202r, the following EL pictures do not rely on incorrect motion vectors on decoder side.
In another embodiment, the same constraint is active for Decoder-side Motion Vector Refinement (DMVR) (i.e. motion vector refinement based on reference sample values), which would otherwise also lead to sever artifacts.
Thus, according to an embodiment, at least one of the following coding concepts is disabled for coding the yet un-refreshed sub-areas 101u of the one or more subsequent pictures 1012, . . . , 101n contained in the refresh period-RP:
According to a further embodiment, at least one of the following coding concepts is disabled for coding the yet un-refreshed sub-area 101u of the GDR picture 103:
and wherein DMVR is disabled for coding the yet un-refreshed sub-areas 101u of the one or more subsequent pictures 1012, . . . , 101n contained in the refresh period RP.
In another embodiment, the coded layer with GDR is coded independent of other layers and it is expressed in the bitstream that sample substitution can be carried out using an indicated other layer with adequate content for sample substitution. Thus, according to this embodiment, the second layer 602 may be coded independently from the first layer 601 or from any further layers, and wherein, if the second sequence 100 of pictures 1011, 1012, . . . , 101n is randomly accessed at the GDR picture 103, the signalization indicates that the yet un-refreshed sub-area 101u of the GDR picture 103 of the second layer 602 is to be inter-layer predicted from samples 202r of the first layer 601 or of any predetermined (indicated) further layer with adequate content.
In another embodiment, the above restriction may take the form of a bitstream requirement depending on the identified refreshed and non-refreshed region.
Furthermore, according to this aspect, it is further suggested to provide a corresponding encoder, decoder, a method for encoding and a method for decoding.
According to an embodiment, it is suggested to provide a decoder for decoding at least one picture from a multi layered scalable video data stream 600 comprising a first sequence 200 of pictures 2011, 2012, . . . , 201n in a first layer (e.g. a base layer) 601 and a second sequence 100 of pictures 1011, 1012, . . . , 101n in a second layer (e.g. an enhancement layer) 602. The second sequence 100 of pictures 1011, 1012, . . . , 101n in the second layer 602 may comprise at least one Gradual Decoder Refresh-GDR-picture 103 as a start picture and one or more subsequent pictures 1012, . . . , 101n in a refresh period-RP. The decoder may be configured to read from the multi layered scalable video data stream 600 a signalization carrying information about a possibility that a yet un-refreshed sub-area 101u of the GDR picture 103 of the second layer 602 is to be inter-layer predicted from samples 202r of the first layer 601, and wherein the decoder is further configured to, responsive to the signalization:
According to a further embodiment, it is suggested to provide an encoder for encoding at least one picture into a multi layered scalable video data stream 600 comprising a first sequence 200 of pictures 2011, 2012, . . . , 201n in a first layer (e.g. base layer) 601 and a second sequence 100 of pictures 1011, 1012, . . . , 101n in a second layer (e.g. an enhancement layer) 602, wherein the second sequence 100 of pictures 1011, 1012, . . . , 101n in the second layer 602 comprises at least one Gradual Decoder Refresh-GDR-picture 103 as a start picture and one or more subsequent pictures 1012, . . . , 101n in a refresh period-RP. The encoder may be configured to write into the multi layered scalable video data stream 600 a signalization carrying information about a possibility that a yet un-refreshed sub-area 101u of the GDR picture 103 of the second layer 602 is to be inter-layer predicted from refreshed samples 202r of the first layer 601. The signalization may further carry information
According to a further embodiment, it is suggested to provide a method for decoding at least one picture from a multi layered scalable video data stream 600 comprising a first sequence 200 of pictures 2011, 2012, . . . , 201n in a first layer (e.g. base layer) 601 and a second sequence 100 of pictures 1011, 1012, . . . , 101n in a second layer (e.g. an enhancement layer) 602, wherein the second sequence 100 of pictures 1011, 1012, . . . , 101n in the second layer 602 comprises at least one Gradual Decoder Refresh-GDR-picture 103 as a start picture and one or more subsequent pictures 1012, . . . , 101n in a refresh period-RP. The method comprises steps of reading from the multi layered scalable video data stream 600 a signalization carrying information about a possibility that a yet un-refreshed sub-area 101u of the GDR picture 103 of the second layer 602 is to be inter-layer predicted from refreshed samples 202r of the first layer 601. The method may further comprise steps of executing, responsive to the signalization, at least one of the following actions:
According to a further embodiment, it is suggested to provide a method for encoding at least one picture into a multi layered scalable video data stream 600 comprising a first sequence 200 of pictures 2011, 2012, . . . , 201n in a first layer (e.g. base layer) 601 and a second sequence 100 of pictures 1011, 1012, . . . , 101n in a second layer (e.g. enhancement layer) 602, wherein the second sequence 100 of pictures 1011, 1012, . . . , 101n in the second layer 602 comprises at least one Gradual Decoder Refresh-GDR-picture 103 as a start picture and one or more subsequent pictures 1012, . . . , 101n in a refresh period (RP). The method comprises steps of writing into the multi layered scalable video data stream 600 a signalization carrying information about a possibility that a yet un-refreshed sub-area 101u of the GDR picture 103 of the second layer 602 is to be inter-layer predicted from samples 202r of the first layer 601. The signalization may further contain information that in yet un-refreshed sub-areas 101u of the one or more subsequent pictures 1012, . . . , 101n contained in the refresh period-RP, motion vector prediction is disabled or motion vector prediction is realized non-temporally, or that in a yet un-refreshed sub-area 101u of the GDR picture 103 motion vector prediction is disabled or motion vector prediction is realized non-temporally.
Scalable video has many benefits for a lot of streaming systems. For instance, by using the correlation of different versions of the content and/or different resolutions, the overall compression efficiency compared to several independent bitstreams (simulcast) with different versions and/or different resolutions is increased drastically. This can lead to big savings on storage at servers and CDNs, reducing the deployment costs of streaming services.
However, the coding efficiency of transmitted scalable bitstream to the end device is lower than the corresponding one if it would be a single layer bitstream. That is, the inter-layer prediction comes along with an efficiency loss due to some signaling overhead. Ideally, a joint optimization should be performed that reduces the required storage capacity of the several versions, i.e. the overall bitrate of all version, conditioned to not increasing drastically the size of each version compared to the single layer version. This could be achieved by evaluating the described optimization problem and marking per AU whether the AU uses inter-layer prediction or not.
In the previous standards this has been done by marking those pictures as discardable with a discardable flag in lower layers. Alternatively, a slice could also mark that inter-layer is not used.
However, those mechanisms are not very flexible and hinder efficient file format usage by facilitating independency in terms of layer dependency of some pictures in the stream, e.g. being able to efficiently drop layers as in the example shown in
Each of the first and second layers 701, 702 may comprise two or more temporal sublayers. For example, the first layer 701 may comprise a first temporal sublayer 701a and a second temporal sublayer 701b. The second layer 702 may comprise a first temporal sublayer 702a and a second temporal sublayer 702b.
Some pictures contained in a layer 701, 702 may be intra coded, as exemplarily depicted with arrows 710. For example, a picture 2012 of a temporal sublayer 701b of a layer 701 may be intra-coded by referencing one or more pictures 2011, 2013 of a different temporal sublayer 701b of the same layer 701.
Alternatively, some other pictures contained in a layer 701, 702 may be inter-coded, as exemplarily indicated with arrows 711. For example, a picture 1011 of a temporal sublayer 702a of a layer 702 may be inter-coded by referencing a picture 2011 of a temporal sublayer 701a of a different layer 701. For instance, a picture 1011 of a first temporal sublayer 702a of the second layer (e.g. enhancement layer) 702 may be inter-coded by referencing a picture 2011 of a first temporal sublayer 701a of the first layer (e.g. base layer) 701.
According to an embodiment, a picture contained in a certain temporal sublayer of a layer may only reference pictures contained in a temporal sublayer of a different layer but with the same temporal sublayer hierarchy. For example, a picture 2011 contained in the first temporal sublayer 702a of the second layer 702 may only reference a picture 1011 that is also contained in the first temporal sublayer 701a but of the first layer 701. That is, the first temporal sublayers 701a, 702a have the same temporal sublayer hierarchy.
In the non-limiting example shown in
Any temporal sublayers that may not be used for inter-coding may be dropped. In particular, any temporal sublayers having a higher temporal sublayer hierarchy than a predetermined temporal sublayer which is used for inter-coding, may be dropped.
Thus, as shown in the non-limiting example of
Mostly, it can be seen that the signaling overhead is more detrimental for higher temporal sublayers in a bitstream 700. E.g., an efficient way of implementing the described feature would be, for example, to encode every second picture as being dependent or e.g. every fourth picture as using inter-layer dependency (note: a picture may be referred to as being dependent if coded with inter-layer dependency, and as being independent if coded without inter-layer dependency).
Therefore, in an embodiment the layer dependency further indicates whether temporal sublayers within a dependent layer depend on the lower layers or not.
This could be done, for instance, in the VPS (or SPS) as shown below:
vps_sub_layer_independent_flag [i] [j] being equal to 1 specifies that the (temporal) sub-layer with index j contained in the layer with index i does not use inter-layer prediction. vps_sub_layer_independent_flag [i] [j] being equal to 0 specifies that the (temporal) sub-layer with index j contained in the layer with index i may use inter-layer prediction.
Thus, according to an embodiment it is suggested to provide a multi layered scalable video data stream 700 comprising a first sequence 200 of pictures 2011, 2012, . . . , 201n in a first layer (e.g. base layer) 701 and a second sequence 100 of pictures 1011, 1012, . . . , 101n in a second layer (e.g. an enhancement layer) 702, each of the first and second layers 701, 702 comprising a plurality of temporal sublayers 701a, 701b; 702a, 702b. The scalable video data stream 700 may comprise a signalization (e.g. vps_sub_layer_independent_flag [i] [j]) indicating which temporal sublayers 702a, 702b of the second layer (e.g. enhancement layer) 702 may be coded by inter-layer prediction.
According to a further embodiment, it is suggested to provide a corresponding decoder for decoding at least one picture from a multi layered scalable video data stream 700 comprising a first sequence 200 of pictures 2011, 2012, . . . , 201n in a first layer (e.g. a base layer) 701 and a second sequence 100 of pictures 1011, 1012, . . . , 101n in a second layer (e.g. an enhancement layer) 702, each of the first and second layers 701, 702 comprising a plurality of temporal sublayers 701a, 701b; 702a, 702b. The decoder may be configured to decode one or more of the temporal sublayers 701a, 701b; 702a, 702b by using inter-layer prediction based on a signalization (e.g. vps_sub_layer_independent_flag [i] [j]) derived from the scalable video data stream 700, said signalization indicating which temporal sublayers 702a, 702b of the second layer 702 are to be coded by inter-layer prediction.
According to a further embodiment, it is suggested to provide a corresponding encoder for encoding at least one picture into a multi layered scalable video data stream 700 comprising a first sequence 200 of pictures 2011, 2012, . . . , 201n in a first layer (e.g. abase layer) 701 and a second sequence 100 of pictures 1011, 1012, . . . , 101n in a second layer (e.g. an enhancement layer) 702, each of the first and second layers 701, 702 comprising a plurality of temporal sublayers 701a, 701b; 702a, 702b. The encoder may be configured to encode one or more of the temporal sublayers 701a, 701b; 702a, 702b by using inter-layer prediction and to write a signalization (e.g. vps_sub_layer_independent_flag [i] [j]) into the scalable video data stream 700, said signalization indicating which temporal sublayers 702a, 702b of the second layer 702 are coded by inter-layer prediction.
According to a further embodiment, it is suggested to provide a corresponding method for decoding at least one picture from a multi layered scalable video data stream 700 comprising a first sequence 200 of pictures 2011, 2012, . . . , 201n in a first layer (e.g. base layer) 701 and a second sequence 100 of pictures 1011, 1012, . . . , 101n in a second layer (e.g. enhancement layer) 702, each of the first and second layers 701, 702 comprising a plurality of temporal sublayers 701a, 701b; 702a, 702b. The method comprises steps of decoding one or more of the temporal sublayers 702a, 702b by using inter-layer prediction based on a signalization derived from the scalable video data stream 700, said signalization (e.g. vps_sub_layer_independent_flag [i] [j]) indicating which temporal sublayers 702a, 702b of the second layer 702 are to be coded by inter-layer prediction.
According to a further embodiment, it is suggested to provide a corresponding method for encoding at least one picture into a multi layered scalable video data stream 700 comprising a first sequence 200 of pictures 2011, 2012, . . . , 201n in a first layer (e.g. base layer) 701 and a second sequence 100 of pictures 1011, 1012, . . . , 101n in a second layer (e.g. enhancement layer) 702, each of the first and second layers 701, 702 comprising a plurality of temporal sublayers 701a, 701b; 702a, 702b. The method comprises steps of encoding one or more of the temporal sublayers 702a, 702b by using inter-layer prediction based on a signalization derived from the scalable video data stream 700, said signalization (e.g. vps_sub_layer_independent_flag [i] [j]) indicating which temporal sublayers 702a, 702b of the second layer 702 are coded by inter-layer prediction.
As mentioned before, the temporal sublayers may comprise a temporal sublayer hierarchy, e.g. first temporal sublayer, second temporal sublayer, third temporal sublayer, and so on. The temporal sublayer hierarchy of a layer may be indicated by means of a temporal identifier, e.g. a syntax element, contained in the above mentioned signalization in the video data stream 700.
Further above, with reference to
The aforementioned temporal identifier may indicate the threshold, i.e. the temporal sublayer hierarchy up to which inter-layer prediction may be used. Stated the other way around, the temporal identifier may indicate the threshold, i.e. the temporal sublayer hierarchy from which inter-layer prediction may not be used
Thus, according to an embodiment, the signalization in the video data stream 700 may comprise a predetermined temporal identifier (threshold) from which the temporal sublayers 702a, 702b of the second layer 702 may be coded without inter-layer prediction.
That means, those temporal sublayers 702a, 702b of the second layer 702 which comprise a temporal identifier having a value above the predetermined temporal identifier (threshold) are coded without inter-layer prediction. In the example shown in
In turn, those temporal sublayers 702a, 702b of the second layer 702 which comprise a temporal identifier having a value up to or below the predetermined temporal identifier (threshold) are coded with inter-layer prediction. In the example shown in
Accordingly, if the signalization (e.g. vps_sub_layer_independent_flag [i] [j]) may indicate that a temporal sublayer 702b of the second layer 702 does not use a temporal sublayer 701b of a lower layer 701 for inter-layer prediction, this temporal sublayer 701b of the lower layer 701 may be marked as discardable or be discarded or dropped from the multi layered scalable video data stream 700.
In another embodiment an enable flag is included into the VPS to indicate whether the sub_layer_independent_flag is included into the VPS or whether per-default all sublayers are dependent on a lower layer.
Thus, according to an embodiment, multi layered scalable video data stream 700 may further comprise a syntax element (e.g. enable_flag in VPS) for indicating whether the signalization (e.g. vps_sub_layer_independent_flag [i] [j]) is included in the multi layered scalable video data stream 700. If not included, then all temporal sublayers 702a, 702b of the second layer 702 may, per default, depend on a lower layer 701.
As mentioned above, an efficient way of implementing the described feature would be, for example, to encode every second picture as being dependent or e.g. every fourth picture as using inter-layer dependency.
Thus, according to an embodiment, the encoder may be configured to encode a first predetermined row of consecutive pictures (e.g. every second picture) 1012, 1014, . . . , 101n−1 of the second sequence 100 of pictures as being dependent, or to encode a second predetermined row of consecutive pictures (e.g. every fourth picture) of the second sequence 100 of pictures as using inter-layer dependency.
Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.
Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be executed by such an apparatus.
Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software or at least partially in hardware or at least partially in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.
Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.
Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitory.
A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.
A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.
In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.
The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.
The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.
While this disclosure has been described with reference to illustrative embodiments, this description is not intended to be construed in a limiting sense. Various modifications and combinations of the illustrative embodiments, as well as other embodiments of this disclosure, will be apparent to persons skilled in the art upon reference to the description. It is therefore intended that the appended claims encompass any such modifications or embodiments.
| Number | Date | Country | Kind |
|---|---|---|---|
| 19199348.4 | Sep 2019 | EP | regional |
This application is a continuation of U.S. application Ser. No. 17/763,453, filed Mar. 24, 2022, which is the U.S. national phase of International Application No. PCT/EP2020/076616 filed Sep. 23, 2020 which designated the U.S. and claims priority to EP patent application Ser. No. 19/199,348.4 filed Sep. 24, 2019, the entire contents of each of which are hereby incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17763453 | Mar 2022 | US |
| Child | 18766977 | US |
