This application claims priority from European Application No. 15306816.8, entitled “Method and Device for Encoding a HDR Picture,” filed on Nov. 16, 2015, and European Application No. 16305102.2, entitled “Method and Device for Encoding a HDR Picture, filed on Feb. 1, 2016, the contents of which are hereby incorporated by reference in its entirety.
In the following, a method and a device for encoding a HDR picture are disclosed.
For distributing HDR (English acronym of “High Dynamic Range”) video content, it is known to first map the HDR video content to obtain a video content of lower dynamic range, also referred to as a SDR (English acronym of “Standard Dynamic Range”) video content. The document ISO/IEC JTC1/SC29/WG11 MPEG2015/M37285 entitled Candidate Test Model for HEVC extension for HDR and WCG video coding and published in October 2015 discloses a method for encoding HDR and also Wide Gamut video contents.
In step S1042, an inverse EOTF function (EOTF stands for “Electro-Optical Transfer Function”) is applied on the HDR picture to obtain a non-linear HDR signal. In step S1044, the non-linear HDR signal is color transformed, if represented with RGB color values, to obtain a YUV signal (i.e. a signal in a YCbCr color space). In step S1046, the YUV signal is transformed from 444 to 420 format, i.e. the chroma is downsampled. In step S1048, the 420 YUV values are quantized into integer values. In step S1050, an ATF function (ATF stands for “Adaptive Transfer Functionality”) is applied on Y. ATF functionality aims at adjusting dynamic range in the YCbCr domain.
There is thus a need to determine the β0,U( ) and β0,V( ) so that the SDR picture is perceptually of good quality.
A method for coding an HDR picture is disclosed that comprises:
In a specific embodiment, determining color correcting parameters comprises:
A coding device for coding a HDR picture is disclosed that comprises:
In a specific embodiment, the means for determining color correcting parameters comprises:
A coding device comprising a communication interface to access a HDR picture is disclosed. The coding device further comprises at least one processor configured to:
According to a specific embodiment, to determine color correcting parameters comprises:
A non-transitory computer readable medium with instructions stored therein is disclosed. The instructions, upon execution, instruct at least one processor to:
In a specific embodiment, to determine color correcting parameters comprises:
The following embodiments and variants applies to all method, device, non-transitory computer readable medium, etc.
According to a specific characteristic, the color appearance value is a hue or a saturation.
In another specific embodiment, determining an intermediate SDR picture so that a color appearance value is preserved comprises for a current pixel:
In yet another non-limiting embodiment, determining the color correcting parameters from the intermediate SDR picture and from the SDR picture comprises minimizing a distance between chroma components of the SDR picture color corrected and corresponding chroma components of the intermediate SDR picture inside a considered luma sub-range.
In a specific embodiment, color correcting the SDR picture responsive to the color correcting parameters comprises dividing each color component of the SDR picture by a color correcting parameter.
It is to be understood that the figures and descriptions have been simplified to illustrate elements that are relevant for a clear understanding of the present principles, while eliminating, for purposes of clarity, many other elements found in typical encoding and/or decoding devices. It will be understood that, although the terms first and second may be used herein to describe various color components, these color components should not be limited by these terms. These terms are only used to distinguish one color component from another. For example, a first color component could be termed “a component” or “a second color component”, and, similarly, a second color component could be termed “another component” or “a first color component” without departing from the teachings of the disclosure.
The transmitter 100 comprises one or more processor(s) 1000, which may comprise, for example, a CPU, a GPU and/or a DSP (English acronym of Digital Signal Processor), along with internal memory 1030 (e.g. RAM, ROM, and/or EPROM). The transmitter 100 comprises one or more communication interface(s) 1010, each adapted to display output information and/or allow a user to enter commands and/or data (e.g. a keyboard, a mouse, a touchpad, a webcam); and a power source 1020 which may be external to the transmitter 100. The transmitter 100 may also comprise one or more network interface(s) (not shown). Encoder module 1040 represents the module that may be included in a device to perform the coding functions. Additionally, encoder module 1040 may be implemented as a separate element of the transmitter 100 or may be incorporated within processor(s) 1000 as a combination of hardware and software as known to those skilled in the art.
The HDR picture or at least one block of the HDR picture may be obtained from a source. According to different embodiments, the source can be, but is not limited to:
According to different embodiments, the stream may be sent to a destination. As an example, the stream is stored in a remote or in a local memory, e.g. a video memory or a RAM, a hard disk. In a variant, the stream is sent to a storage interface, e.g. an interface with a mass storage, a ROM, a flash memory, an optical disc or a magnetic support and/or transmitted over a communication interface, e.g. an interface to a point to point link, a communication bus, a point to multipoint link or a broadcast network.
According to an exemplary and non-limiting embodiment, the transmitter 100 further comprises a computer program stored in the memory 1030. The computer program comprises instructions which, when executed by the transmitter 100, in particular by the processor 1000, enable the transmitter 100 to execute the method described with reference to any
According to exemplary and non-limiting embodiments, the transmitter 100 can be, but is not limited to:
The method begins at step S200. At step S202, the transmitter 100 access a HDR picture, e.g. a HDR picture of RGB components. At step S204, the transmitter maps the accessed HDR picture to obtain a SDR picture, e.g. a YUV or YCbCr SDR picture. As an example, a simple inverse EOTF function, e.g. the known PQ EOTF, may be applied followed by a color space conversion to generate a YCbCr picture. In yet another variant, the steps S1042 to S1050 disclosed with reference to
It will be appreciated, however, that the present principles are not restricted to these specific methods for mapping a HDR picture in order to obtain a SDR picture. Indeed, any method making it possible to reduce the dynamic range of a HDR picture is suitable. In step S207, the obtained SDR picture is color corrected. More precisely, its color components, e.g. its chroma samples such as UV or CbCr, are modified so that the perceptual quality of the color corrected SDR picture is increased. The step S207 comprises a first step S2070 and a second step S2076.
In the step S2070, the transmitter determines color correcting parameters β0,U( ) and β0,V( ).
In step S3004, a corresponding HDR linearized luminance value LHDR is obtained. The linearized luminance value LHDR is either obtained directly from a memory or is determined from corresponding RGB values according to the following equation:
The matrix M1×3 is for example provided by specification ITU-R BT2020 for a RGB2020 to YCbCr ad-hoc 3×3 color space conversion.
In step S3006, a ratio w linking the SDR and HDR linearized luminance values for the current pixel is determined as follows: w=LSDR/LHDR.
In step S3008, the linear SDR RGB values denoted
for the current pixel are determined from the accessed HDR picture and the ratio w as follows:
where
are the linear RGB values for the co-located pixel in the accessed HDR picture. Finally, in step S3010, the non-linear SDR chroma values (UV or CbCr) are determined from the linear SDR RGB values. The operation consists in SDR gamma correction (approximated by a power of 0.45 here) followed by the RGB to YCbCr color space conversion 3×3 matrix. Since Y is not modified, the first line of the 3×3 matrix is discarded.
The current pixel in the intermediate picture has thus the following values: (Y Uint Vint). The steps S3002 to S3010 are iterated until all pixels of a picture region, of a whole picture or of a group of pictures are processed. Indeed, the method may apply on successive pictures of a video, or on only a portion of a picture.
At step S3012, the method ends the loop over the pixels.
In the step S2074, the transmitters determines the color correcting functions/parameters from the intermediate picture (Y, Uint,Vint) and from the SDR picture (Y,U,V) obtained at step S204, or from regions of such pictures or from successive intermediate pictures and their corresponding SDR pictures.
To this aim, the luma range is divided in a plurality of subranges. In a step S4000, a loop over the subranges begins. In step S4002, the first parameter/function β0,U[ ] is determined for the current subrange denoted currSliceY as follows:
In a step S4004, the second parameters/function β0,V[ ] is determined for the current subrange denoted currSliceY as follows:
β0,U[⋅] is for example determined through a least mean square procedure, which aims at minimizing the L2 distance between the color corrected chroma component U/β0,U[currSlice] and the intermediate Uint component values inside the considered Y sub-range. The optimal β0,U[currSlice] value is given by:
β0,V[⋅] is computed the same way as β0,U[⋅]·β0,V[⋅] and β0,U[⋅] may be represented by Look-Up Tables. The steps S4002 and S4004 are iterated until all subranges are processed.
The loop over subranges ends at step S4006.
In a variant, the transmitter determines the color correcting parameters from several successive intermediate pictures, for example for a GOP (English acronym of “Group Of Pictures”) or from only picture portions.
Note that with the method of
In the step S2076, the transmitters color corrects the obtained SDR picture using the determined color correcting parameters. More precisely, the chroma components of the SDR picture obtained at step S204 are modified as follows:
U=u/β0,U[Y] and V=v/β0,v[Y]. Color correcting thus comprises dividing each chroma component Cb and Cr, noted U and V here, respectively by a factor β0,U[Ycoloc] and β0,V[Ycoloc] which depend on the luma signal at the same spatial position, where β0,U[Ycoloc] and β0,V[Ycoloc] are the function or LUTs determined at step S2074. The encoding method is backward compatible, i.e. the SDR after color correction may be viewed with a good level of consistency with the accessed HDR picture. Further the hue is at least partially preserved between the accessed HDR picture and the color corrected SDR picture.
In a step S208, the color corrected SDR picture is encoded. The color correcting parameters/functions are also encoded for example in SEI message (SEI stands for “Supplemental Enhancement Information”), in a Picture Parameter Set, or any type of aside metadata. The encoding is for example compliant with the HEVC video coding standard as disclosed in the document from ITU-T entitled “H.265 series H: Audiovisual and Multimedia systems Infrastructure of audiovisual services—Coding of moving video”, MPEG-2, or H.264, etc. It will be appreciated, however, that the present principles are not restricted to these specific methods for encoding a picture. Indeed, any method making it possible to encode a picture is suitable. Encoding a picture usually comprises dividing the picture into picture blocks. The encoding method applies on each block. For a current block, the encoding method usually comprises determining a prediction, determining a residual block by subtracting the prediction block from the current block. The residual block is then transformed into frequency coefficients, e.g. by a Discrete Cosine Transform. The frequency coefficients are then quantized and the quantized coefficients are entropy coded. These steps are found in typical encoding and/or decoding methods.
The method disclosed with the preservation of the hue may be generalized to the preservation of any appearance value. In another embodiment, the saturation may be preserved instead of the hue or in addition to the hue.
The implementations described herein may be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method or a device), the implementation of features discussed may also be implemented in other forms (for example a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented in, for example, an apparatus such as, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable/personal digital assistants (“PDAs”), and other devices that facilitate communication of information between end-users.
Implementations of the various processes and features described herein may be embodied in a variety of different equipment or applications, particularly, for example, equipment or applications. Examples of such equipment include an encoder, a decoder, a post-processor processing output from a decoder, a pre-processor providing input to an encoder, a video coder, a video decoder, a video codec, a web server, a set-top box, a laptop, a personal computer, a cell phone, a PDA, and other communication devices. As should be clear, the equipment may be mobile and even installed in a mobile vehicle.
Additionally, the methods may be implemented by instructions being performed by a processor, and such instructions (and/or data values produced by an implementation) may be stored on a processor-readable medium such as, for example, an integrated circuit, a software carrier or other storage device such as, for example, a hard disk, a compact diskette (“CD”), an optical disc (such as, for example, a DVD, often referred to as a digital versatile disc or a digital video disc), a random access memory (“RAM”), or a read-only memory (“ROM”). The instructions may form an application program tangibly embodied on a processor-readable medium. Instructions may be, for example, in hardware, firmware, software, or a combination. Instructions may be found in, for example, an operating system, a separate application, or a combination of the two. A processor may be characterized, therefore, as, for example, both a device configured to carry out a process and a device that includes a processor-readable medium (such as a storage device) having instructions for carrying out a process. Further, a processor-readable medium may store, in addition to or in lieu of instructions, data values produced by an implementation.
As will be evident to one of skill in the art, implementations may produce a variety of signals formatted to carry information that may be, for example, stored or transmitted. The information may include, for example, instructions for performing a method, or data produced by one of the described implementations. For example, a signal may be formatted to carry as data the rules for writing or reading the syntax of a described embodiment, or to carry as data the actual syntax-values written by a described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries may be, for example, analog or digital information. The signal may be transmitted over a variety of different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.
A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made. For example, elements of different implementations may be combined, supplemented, modified, or removed to produce other implementations. Additionally, one of ordinary skill will understand that other structures and processes may be substituted for those disclosed and the resulting implementations will perform at least substantially the same function(s), in at least substantially the same way(s), to achieve at least substantially the same result(s) as the implementations disclosed. Accordingly, these and other implementations are contemplated by this application.
Number | Date | Country | Kind |
---|---|---|---|
15306816 | Nov 2015 | EP | regional |
16305102 | Feb 2016 | EP | regional |
Number | Name | Date | Kind |
---|---|---|---|
20130241931 | Mai | Sep 2013 | A1 |
20150201222 | Mertens | Jul 2015 | A1 |
20170026646 | Minoo | Jan 2017 | A1 |
20170078706 | Van Der Vleuten | Mar 2017 | A1 |
20180232867 | Park | Aug 2018 | A1 |
Number | Date | Country |
---|---|---|
3051792 | Aug 2016 | EP |
Entry |
---|
Lasserre et al., “Technicolor's response to CfE for HDR and WCG (category 1)—Single layer HDR video coding with SDR backward compatibility”, International Organisation for Standardisation Organisation Internationale de Normalisation ISO/IEC JTC1/SC29/WG11 Coding of Moving Pictures and Audio, ISO/IEC JTC1/SC29/WG11 MPEG2014/M36263r1, Warsaw, Poland, Jun. 2015, pp. 1-21. |
Pouli et al., “Color Correction for Tone Reproduction”, 21st Color and Image Conference, Nov. 2013, pp. 215-220. |
Mantiuk et al., “Color correction for tone mapping”, Eurographics 2009, vol. 28, No. 2, Apr. 2009, pp. 193-202. |
Baylon et al., “Response to Call for Evidence for HDR and WCG Video Coding: Arris, Dolby and InterDigital”, International Organisation for Standardisation Organisation Internationale de Normalisation ISO/IEC JTC1/SC29/WG11 Coding of Moving Pictures and Audio, ISO/IEC JTC1/SC29/WG11 MPEG2015/M36264, Warsaw, Poland, Jun. 2015, pp. 1-9. |
Pascale, “A Review of RGB Color Spaces . . . from xyY to R′G′B′”, The BabelColor Company, Montreal, Canada, Oct. 6, 2003, pp. 1-35. |
Anonymous, “Parameter values for ultra-high definition television systems for production and international programme exchange”, International Telecommunication Union Standard, ITU-R Radiocommunication Sector of ITU, Recommendation ITU-R BT.2020-1, BT Series Broadcasting service (television), Jun. 2014, pp. 1-8. |
Anonymous, “Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, High efficiency video coding”, International Telecommunication Union Standard, ITU-T Telecommunication Standardization Sector of ITU, Recommendation ITU-T H.265, Oct. 2014, pp. 1-540. |
Anonymous, “Candidate Test Model for HEVC extension for HDR and WCG video coding”, International Organisation for Standardisation Organisation Internationale de Normalisation ISO/IEC JTC1/SC29/WG11 Coding of Moving Pictures and Audio, ISO/IEC JTC1/SC29/WG11 MPEG2015/M37xxx, Geneva, Switzerland, Oct. 2015, pp. 1-10. |
Anonymous, “High Dynamic Range Electro-Optical Transfer Function of Mastering Reference Displays”, FCD SMPTE Standard, SMPTE ST2084:201X, Version 1.24, Jun. 18, 2014, pp. 1-14. |
Anonymous, “CIE 1931 color space”, Wikipedia, https://en.wikipedia.org/wiki/CIE_1931_color_space, Sep. 10, 2016, pp. 1-10. |
Anonymous, “Series H: Audiovisual and Multimedia Systems, Infrastructure of audiovisual services—Coding of moving video, Advanced video coding for generic audiovisual services”, International Telecommunication Union Standard, ITU-T Telecommunication Standardization Sector of ITU, Recommendation ITU-T H.264, May 2003, pp. 1-282. |
Anonymous, “Information technology—Generic coding of moving pictures and associated audio information: Video”, International Standard ISO/IEC 13818-2, May 15, 1996, pp. 1-212. |
Number | Date | Country | |
---|---|---|---|
20170142446 A1 | May 2017 | US |
Number | Date | Country | |
---|---|---|---|
15353045 | Nov 2016 | US |