Patent Application
20180220190
In the following, a method for decoding a video and a decoding device are disclosed. Corresponding encoding method and encoding device are further disclosed.
A color gamut is a certain complete set of colors in a color space. A color gamut is typically defined in a given color space (e.g. CIE xyY color space) by color primaries (in general three primaries, which results in a triangle in a chromaticity diagram) and a white point.
The deployment of WCG (English acronym of Wide Color Gamut) video is envisioned in the short-term future. However, not all the rendering devices will be able to render WCG content. Therefore, a gamut mapping process, also known as color mapping process, may be required to convert the WCG content into a color gamut adapted to the rendering device. P3/BT.2020 are examples of WCG while BT.709 is an example of a narrower color gamut. It is expected that most of the content produced in the mid-term future will be BT.2020 or P3 content, i.e. a content represented in a BT.2020 or P3 colorgamut. For the purpose of backward compatibility with BT.709 devices, the color mapping of a P3 content into a BT.709 content is desirable while keeping the ability to restore the BT.2020 content from the BT.709 content. The color mapping process has thus to be invertible. A color mapping process usually makes use of color mapping information or color mapping data. An example of such color mapping information is the Supplemental Enhancement Information (SEI) message defined in sections D.2.32 and D.3.32 of the document ITU-T H.265 (also known as HEVC video coding standard) published in April 2015. The color mapping information of section D.3.32 defines color mapping functions such 1D LUTs (English acronym of Look-Up Table) and 3×3 matrix. This color mapping information may be content dependent and transmitted for example with an encoded BT.709 video content so that one is able to reconstruct a BT.2020 video content from the decoded BT.709 video content and the color mapping information. However, transmitting such color mapping information may be costly in terms of bitrate especially in the context of video broadcasting.
According to an aspect of the present principles, a method is disclosed comprising obtaining a video from a stream; obtaining information representative of a version of color mapping information; determining color mapping information responsive to said information representative of a version; and color mapping the obtained video with said determined color mapping information.
The present embodiments also provide a device comprising means for obtaining a video from the at least one stream; means for obtaining information representative of a version of color mapping information; means for determining a color mapping information responsive to said information representative of a version; and means for color mapping the obtained video with said determined color mapping information.
Advantageously, obtaining information representative of a version of color mapping information comprises decoding an index from a stream indicating a version of color mapping information.
The present embodiments also provide a device comprising a communication interface configured to access at least one stream and at least one processor configured to obtain a video from the at least one stream; obtain information representative of a version of color mapping information; determine color mapping information responsive to said information representative of a version; and color map the obtained video with said determined color mapping information.
Advantageously, to obtain information representative of a version of color mapping information comprises decoding an index from a stream indicating a version of color mapping information.
The present embodiments also provide a computer program product comprising program code instructions to execute the following steps when this program is executed on a computer: obtaining a video from a stream; obtaining information representative of a version of color mapping information; determining color mapping information responsive to said information representative of a version; and color mapping the obtained video with said determined color mapping information.
The present embodiments also provide a non-transitory computer readable medium with instructions stored therein which, upon execution, instruct at least one processor to: obtain a video from the at least one stream; obtain information representative of a version of color mapping information; determine color mapping information responsive to said information representative of a version; and color map the obtained video with said determined color mapping information.
According to another aspect of the present principles, a method is disclosed that comprises encoding a video from a stream; obtaining information representative of a version of color mapping information; encoding said information representative of a version.
The present embodiments also provide a device comprising means for encoding a video from a stream; means for obtaining information representative of a version of color mapping information; and means for encoding said information representative of a version.
Advantageously, encoding said information representative of a version of color mapping information comprises encoding an index in a stream indicating a version of color mapping information.
The present embodiments also provide a device comprising a communication interface configured to access at least one video and at least one processor configured to encode the at least one video in a stream; obtain information representative of a version of color mapping information; and encode said information representative of a version.
Advantageously, to encode said information representative of a version of color mapping information comprises encoding an index in a stream indicating a version of color mapping information.
The present embodiments also provide a non-transitory computer readable medium with instructions stored therein which, upon execution, instruct at least one processor to: encode a video from a stream; obtain information representative of a version of color mapping information; and encode said information representative of a version.
The present embodiments also provide a computer program product comprising program code instructions to execute the following steps when this program is executed on a computer: encoding a video from a stream; obtaining information representative of a version of color mapping information; encoding said information representative of a version.
The present embodiments also provide a stream or a storage medium tangibly embodying this stream, wherein the stream comprises at least:
information representative of a video; and
information representative of a version of color mapping information.
In the following, the word “reconstructed” and “decoded” can be used interchangeably. It will be appreciated that the present principles are not restricted to a video and can be applied to a single picture.
The color mapping information used by the color mapping module 114 may be determined dynamically on the transmitter side so that it is fully adapted to the content. The colour_remapping_information SEI message of D.2.32 and D.3.32 in ITU-T H.265 may be used to transmit such information. However, in the case of broadcast applications the transmission of such dynamic color mapping information may be costly in terms of bit rate. Also having completely dynamic metadata may be lead to complex receiver implementations. The present principles are directed to a method and a device for reducing the cost and the complexity of the receiver. Therefore, the color mapping information is not transmitted in a stream but color mapping information is stored in a memory of the receiver 150.
The receiver 150 may be connected to an external device (rendering device) 200 via an output 195. The rendering device is able to provide information related to its capabilities (supported color gamut, chroma format, chroma sampling format . . . ) to the video receiver 150. The video receiver 150 produces and provides a reconstructed video to the external device 200. It is considered that the external device supports a color gamut OCG that is different from the input color gamut ICG. When both color gamuts are identical, no specific adaptation (color mapping) process is needed. The receiver 150 comprises a video decoder 170 configured to decode the video stream and thus to obtain a reconstructed video. The reconstructed video is transmitted to a color mapping module 190 configured to color map the reconstructed video based on color mapping information CRI[k]. The receiver 150 also comprises an identification module 180 configured to identify color mapping information responsive to the color information received on the second input 90 and possibly responsive to user preferences. The color mapping information CRI[k] is identified among several color mapping information stored in a memory 160.
The main advantage is to allow backward compatibility with rendering devices supporting lower color gamut than the color gamut of the native content, while preserving the good reconstruction at the receiver side of the content in its native color gamut, for rendering devices supporting this native color gamut. In addition, the present principles enable the re-use the CRI mechanism for the inverse gamut mapping, which allows a more efficient implementation of the inverse gamut mapping process. For instance, since the CRIs set is defined, the implementation can be accurately optimized for these specific CRIs (e.g. in an hardware implementation). This also avoids or limits the transmission in the stream of content-dependent and dynamic CRI metadata.
At step S10, a video is obtained. For example, the video is decoded from a stream F by the video decoder 170, for example, using an AVC or HEVC video decoder. The video can be a portion of a large video. The video may comprise a single picture. It will be appreciated, however, that the present principles are not restricted to this specific AVC and HEVC decoder. Any video decoder can be used. The output of step S10 is a reconstructed video in an Input Color Gamut (ICG), e.g. BT.709.
At step S12, information is obtained that is representative of one version of color mapping information. This information may be obtained by the identification module 180. It makes it possible to identify one version of color mapping information among at least two versions. Different versions of color mapping information favor the preservation of different sets of colors during the color mapping process. A version of color mapping information is defined at least for a given input color gamut and a given output color gamut as depicted on
In a specific and non-limiting embodiment, the information for identifying a version of color mapping information is decoded from the stream F or from another stream. The information can be an index identifying the version. The index may be a binary flag in the case of two versions are considered as depicted in Table 1. It will be appreciated that the present principles are not restricted to two versions.
In this table, the following parameters are introduced:
The index is not necessarily a binary flag as depicted in table 2.
In the case of more than two color mapping information items mapping from different input and/or output color gamuts, a binary flag may be sufficient if only two versions exist for given input and output color gamuts as depicted in table 3.
In the three tables above, the output color gamut and the mapping color gamut are identical. In a variant depicted on Table 4, they are different. In this case, the «output color gamut» is the color gamut in which the color samples of the output video content are represented (coded). The «mapping gamut» is the color gamut in which the color samples of the input are mapped. The mapped color samples are inside this mapped color gamut, even if they are represented (coded) in an output color gamut that may be different from the mapped color gamut. For instance, in the case of a P3 video content encapsulated in a BT.2020 container, the output color gamut is the BT.2020 gamut, while the mapped color gamut is P3. In this case also, the index may be a non-binary index (from 0 to 7).
In another variant depicted on Table 5, input and output transfer functions may also be taken into account. The «input transfer function» is the transfer function used to code the input video content. The «output transfer function» is the transfer function used to code the output video content. These transfer functions can correspond to a Gamma function with a parameter of 2.2 (G.2.2) or to a perceptual quantization (PQ) transfer function.
In this case also, the index may be a non-binary index (from 0 to 11). In a variant, the version can be further determined via user preference. In such a case the version selected by the user may overwrite the version obtained from the stream. The user can also provide preferred settings for these different parameters, via user preferences. For instance the user may indicate using the “version” parameter if he prefers preserving the BT.709 gamut, or if he prefers having a better preservation of colors outside the BT.709 gamut.
At step S14, color mapping information is determined responsive to the information identifying a version and obtained at step S12. The color mapping information may be determined by the identification module 180 of the receiver 150. As an example, the different versions of color mapping information are stored in a memory, e.g. the memory 160 of
As an example with reference to the table 1 above in the case where an index 1 is decoded the version B of color mapping information is determined that maps from BT.709 to P3 with input and output color formats Y′CbCr.
At step S16, the obtained/decoded video obtained at step S10 is color mapped using the color mapping information determined at step S14. The obtained/decoded video is color mapped for example in the color mapping module 190 of the receiver 150. The mapped video may be stored or further transmitted for display, e.g. via output 195 of receiver 150. The following equations illustrate the application of color mapping information to (R,G,B) values of a color sample in the case where it is defined as a combination of three 1D LUTs (LUTin0, LUTin1, LUTin2) followed by a 3×3 matrix M3×3 followed by three 1D LUTs (LUTout0, LUTout1, LUTout2):
where (Rin, Gin, Bin) are input color values and (Rout, Gout, Bout) are output color values, i.e. after the color mapping.
The method starts at step S200. At step S210, a receiver accesses one or more stream(s). At step S220, the receiver decodes the video from a stream F using for example an AVC or HEVC video decoder. To this aim, the decoder may loop over individual pictures in the input video. At step S230, the receiver decodes a syntax element used_CRI_version from a stream (stream F or another stream) representative of a version of color mapping information. Syntax element used_CRI_version makes it possible to indicate which version of color mapping information has to be used in the case where several color mapping information items correspond to the same «input color gamut», «output/mapping color gamut». It corresponds to the «version» parameter of tables 1 to 5. Advantageously, used_CRI_version is a binary flag in the case of 2 versions.
At step S240 an input color gamut is obtained. The input color gamut is for example decoded from a stream that may be the stream F or another stream (side metadata). This information is typically provided in the VUI. At step S250 an output color gamut is obtained. The output color gamut is for example obtained from the rendering device information. In a variant (not represented), a mapping color gamut is further obtained. It is used with Tables 4 and 5. This information is typically provided in the MDCV SEI message. At step S260, the receiver begins to loop over the color mapping information items CRI[i] stored in a memory of the receiver. At step S270, the receiver checks whether the obtained input color gamut is identical to a CRI[i] input color gamut. If the obtained input color gamut is identical to the CRI[i] input color gamut, the receiver further checks S280 whether the obtained output color gamut is identical to a CRI[i] output color gamut. Otherwise, if the obtained input color gamut is different from the CRI[i] input color gamut, the receiver checks whether more color mapping information items CRI[i] are stored at step S290. If yes, the control returns to step S270. Otherwise, method ends at step S320. If the obtained output color gamut is identical to the CRI[i] output color gamut, the receiver further checks S300 whether used_CRI_version is identical to a CRI[i] version. Otherwise, if the obtained output color gamut is different from the CRI[i] output color gamut, the receiver checks whether more color mapping information items CRI[i] are stored at step S290. If yes, the control returns to step S270. Otherwise, method ends at step S320. If used_CRI_version is identical to the CRI[i] version, the receiver color maps the decoded video with the color mapping information CRI[i] at step S310 and method ends at step S320. Otherwise, if used_CRI_version is different from the CRI[i] version, the receiver checks whether more color mapping information items CRI[i] are stored at step S290. If yes, the control returns to step S270. Otherwise, method ends at step S320.
In variants (not represented), other information may be used. As an example, the receiver may further check whether an obtained input color format is identical to a CRI[i] input color format. The input color format may be obtained from a stream. This information is typically provided in the VUI. The receiver may also check whether an obtained output color format is identical to a CRI[i] output color format. The output color format is for example obtained from the rendering device information. In another example, the receiver may checks whether an obtained input transfer function is identical to a CRI[i] input transfer function and whether an obtained output transfer function is identical to a CRI[i] output transfer function. The input transfer function is for example obtained from a stream, e.g. directly derived from on OETF (English acronym of Opto-Electrical Transfer Function). This information is typically provided in the VUI. The output transfer function may be obtained from the rendering device information. It is the same as the OETF supported by the external device.
Other information that is useful to the receiver to identify the color mapping information to be applied is the color gamut and the chroma format supported by the external device. They can be used to identify the “output color format”, “output color gamut” and “output transfer function” of tables 1 to 5.
Other information such as the supported bit-depth of the different components of the color samples can also be useful.
The receiver 150 comprises one or more processor(s) 1100, which could comprise, for example, a CPU, a GPU and/or a DSP (English acronym of Digital Signal Processor), along with internal memory 1130 (e.g. RAM, ROM and/or EPROM). The receiver 150 comprises one or more communication interface(s) 1110, each adapted to display output information and/or allow a user to enter commands and/or data (e.g. a keyboard, a mouse, a touchpad, a webcam); and a power source 1120 which may be external to the receiver 150. The receiver 150 may also comprise one or more network interface(s) (not shown). Decoder module 1140 represents the module that may be included in a device to perform the decoding functions. Additionally, decoder module 1140 may be implemented as a separate element of the receiver 150 or may be incorporated within processor(s) 1100 as a combination of hardware and software as known to those skilled in the art.
The stream may be obtained from a source. According to different embodiments, the source can be, but not limited to:
According to different embodiments, the decoded video may be sent to a destination, e.g. a display device. As an example, the decoded video is stored in a remote or in a local memory, e.g. a video memory or a RAM, a hard disk. In a variant, the decoded video is sent to a storage interface, e.g. an interface with a mass storage, a ROM, a flash memory, an optical disc or a magnetic support and/or transmitted over a communication interface, e.g. an interface to a point to point link, a communication bus, a point to multipoint link or a broadcast network.
According to an exemplary and non-limiting embodiment, the receiver 150 further comprises a computer program stored in the memory 1130. The computer program comprises instructions which, when executed by the receiver 150, in particular by the processor 1100, enable the receiver to execute the method described with reference to
According to exemplary and non-limiting embodiments, the receiver 150 can be, but not limited to:
At step S20, a video is encoded in a stream F, for example, using an AVC or HEVC video encoder. The video can be a portion of a large video. The video may comprise a single picture. It will be appreciated, however, that the present principles are not restricted to this specific AVC and HEVC encoder. Any video encoder can be used. The video may be a color map version of a master video.
At step S22, information is obtained that is representative of a version of color mapping information. It makes it possible to identify a version of color mapping information. As an example, the information is specified by a user, e.g. by a director of photography.
At step S24, the information obtained at step S30 is encoded in a stream that may be the stream F.
The transmitter 100 comprises one or more processor(s) 1000, which could comprise, for example, a CPU, a GPU and/or a DSP (English acronym of Digital Signal Processor), along with internal memory 1030 (e.g. RAM, ROM, and/or EPROM). The transmitter 100 comprises one or more communication interface(s) 1010, each adapted to display output information and/or allow a user to enter commands and/or data (e.g. a keyboard, a mouse, a touchpad, a webcam); and a power source 1020 which may be external to the transmitter 100. The transmitter 100 may also comprise one or more network interface(s) (not shown). Encoder module 1040 represents the module that may be included in a device to perform the coding functions. Additionally, encoder module 1140 may be implemented as a separate element of the transmitter 100 or may be incorporated within processor(s) 1000 as a combination of hardware and software as known to those skilled in the art.
The video may be obtained from a source. According to different embodiments, the source can be, but is not limited to:
According to different embodiments, the stream may be sent to a destination. As an example, the stream is stored in a remote or in a local memory, e.g. a video memory or a RAM, a hard disk. In a variant, the stream is sent to a storage interface, e.g. an interface with a mass storage, a ROM, a flash memory, an optical disc or a magnetic support and/or transmitted over a communication interface, e.g. an interface to a point to point link, a communication bus, a point to multipoint link or a broadcast network.
According to an exemplary and non-limiting embodiment, the transmitter 100 further comprises a computer program stored in the memory 1030. The computer program comprises instructions which, when executed by the transmitter 100, in particular by the processor 1000, enable the transmitter 100 to execute the method described with reference to
According to exemplary and non-limiting embodiments, the transmitter 100 can be, but is not limited to:
The implementations described herein may be implemented in, for example, a method or a process, a device, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method or a device), the implementation of features discussed may also be implemented in other forms (for example a program). A device may be implemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented in, for example, a device such as, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable/personal digital assistants (“PDAs”), and other devices that facilitate communication of information between end-users.
Implementations of the various processes and features described herein may be embodied in a variety of different equipment or applications, particularly, for example, equipment or applications. Examples of such equipment include an encoder, a decoder, a post-processor processing output from a decoder, a pre-processor providing input to an encoder, a video coder, a video decoder, a video codec, a web server, a set-top box, a laptop, a personal computer, a cell phone, a PDA, and other communication devices. As should be clear, the equipment may be mobile and even installed in a mobile vehicle.
Additionally, the methods may be implemented by instructions being performed by a processor, and such instructions (and/or data values produced by an implementation) may be stored on a processor-readable medium such as, for example, an integrated circuit, a software carrier or other storage device such as, for example, a hard disk, a compact diskette (“CD”), an optical disc (such as, for example, a DVD, often referred to as a digital versatile disc or a digital video disc), a random access memory (“RAM”), or a read-only memory (“ROM”). The instructions may form an application program tangibly embodied on a processor-readable medium. Instructions may be, for example, in hardware, firmware, software, or a combination. Instructions may be found in, for example, an operating system, a separate application, or a combination of the two. A processor may be characterized, therefore, as, for example, both a device configured to carry out a process and a device that includes a processor-readable medium (such as a storage device) having instructions for carrying out a process. Further, a processor-readable medium may store, in addition to or in lieu of instructions, data values produced by an implementation.
As will be evident to one of skill in the art, implementations may produce a variety of signals formatted to carry information that may be, for example, stored or transmitted. The information may include, for example, instructions for performing a method, or data produced by one of the described implementations. For example, a signal may be formatted to carry as data the rules for writing or reading the syntax of a described embodiment, or to carry as data the actual syntax-values written by a described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries may be, for example, analog or digital information. The signal may be transmitted over a variety of different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.
A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made. For example, elements of different implementations may be combined, supplemented, modified, or removed to produce other implementations. Additionally, one of ordinary skill will understand that other structures and processes may be substituted for those disclosed and the resulting implementations will perform at least substantially the same function(s), in at least substantially the same way(s), to achieve at least substantially the same result(s) as the implementations disclosed. Accordingly, these and other implementations are contemplated by this application.
| Number | Date | Country | Kind |
|---|---|---|---|
| 15306172.6 | Jul 2015 | EP | regional |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2016/065934 | 7/6/2016 | WO | 00 |
