The present application is concerned with chroma subsampling such as a chroma subsampling of HDR images.
Traditional low dynamic range (LDR) image and video coding schemes utilize color spaces like YCbCr with one luminance and two chrominance channels and subsampling of the chrominance channels to exploit the fact that the human eye is less sensitive to chroma variations than to luminance variations.
For high dynamic range (HDR) video an adaptive LogLuv color space has been proposed in [1]. Here it is possible to subsample the u and v chrominance channels as well.
However, especially in HDR images and video, edges with every high contrast may occur, i.e. areas with very high luminance next to areas with very low luminance. At such edges, artifacts due to the chroma subsampling can become visible.
FIG. A shows a tone map HDR image after a chroma subsampling with filters according to [2] and upsampling with filters according to [3]. Color bleeding artifacts are clearly visible at portions highlighted with a rectangle and indicated in an enlarged manner beside the image, namely at the rim of the flower pot and between the blades of the blind. These artifacts basically occur because chroma values from dark and bright regions get mixed-up during the process of chroma subsampling and upsampling.
According to an embodiment, an apparatus for chroma subsampling may have: a detector configured to detect high contrast areas in a luminance channel of an image to be chroma subsampled so as to partition the image into a first region composed of the high contrast areas and a second region distinct from the first region; a first chroma subsampler configured to chroma subsample the image in the first region; a second chroma subsampler configured to chroma subsample the image in the second region, wherein the first chroma subsampler exhibits a higher edge preserving property than compared to the second chroma subsampler.
According to another embodiment, a method for chroma subsampling may have the steps of: detecting high contrast areas in a luminance channel of a image to be chroma subsampled so as to partition the image into a first region composed of the high contrast areas and a second region distinct from the first region; chroma subsample the image in the first region using a first chroma subsampler; chroma subsample the image in the second region a second chroma subsampler, wherein the first chroma subsampler exhibits a higher edge preserving property than compared to the second chroma subsampler.
According to another embodiment, a computer program may have a program code for performing, when running on a computer, a method according to claim 11.
It is a basic idea of the present invention that a chroma subsampling having reduced artifacts may be achieved by detecting high contrast areas in a luminance channel of an image to be chroma subsampled so as to partition the image into a first region composed of the high contrast areas and a second region distinct from the first region, with chroma subsampling the image in the first region using a first chroma subsampler and chroma subsampling the image in the second region using a second chroma subsampler, with the first chroma subsampler having a higher edge preserving property than compared to the second chroma subsampler. Thereby, bleeding artifacts may be avoided at least partially, while other characteristics of the image such as the saturation of the image, as otherwise influenced by using rank filters globally, for example, may be substantially preserved.
Embodiments of the present invention will be detailed subsequently referring to the appended drawings, in which:
Before describing some embodiments of the present application, the nature of chroma subsampling is described with respect to
Each “sample” has associated therewith a certain spatial sampled position and a sample value. Merely the samples' sample positions are indicated in
In the example of
The operations which lead from the chroma sampling pattern according to
In order to overcome the problems outlined above in the introductory portion of the specification of the present application, i.e. the color bleeding artifacts associated with the usually used chroma subsampling processes, the inventors of the present application had the following idea. In particular, color bleeding artifacts may be avoided if only chroma values from the bright areas, for example, are used for the subsampling process. For example, an easy way to realize this would be to find the position of the highest luminance value in a 2×2 pixel area for 4:2:0, or in a 2×1 pixel area for 4:2:2, and use the chrominance values from the co-located positions of the respective pixel areas as subsampled chrominance values for this 2×2, or 2×1, pixel area. However, if this would be done globally for the whole image/picture, another artifact occurs: the overall color of the image gets de-saturated. Thus, two methods for chroma subsampling are used in accordance with the embodiments outlined below: a chroma subsampling method that avoids the color bleeding artifact, and another chroma subsampling method that avoids the de-saturation artifact. A method that avoids the de-saturation would be, for example, a down-conversion filter according to [2] or a simple arithmetic mean of four chroma values of a 2×2 pixel area for 4:2:0, for example. Accordingly, the embodiments outlined further below use a determination as to where which of the two chroma subsampling methods should be applied. For example, a simple method for this decision is, as further outlined below, to find the highest and the lowest luminance value in a 2×2 (in case of 4:2:0) or 2×1 (in case of 4:2:2) pixel area. If the ratio between the highest and the lowest value exceeds a threshold, the color bleeding avoiding method is applied, otherwise the de-saturation avoiding chroma subsampling method is applied. Experimentally, a threshold of a ratio of two has been found to give good results, but naturally this is just an example and may depend on the circumstance and may be varied accordingly.
The first chroma subsampler 14 and second chroma subsampler 16 differ from each other in that the first chroma subsampler has a higher edge preserving property than compared to the second chroma subsampler 16. For example, the first chroma subsampler could be configured to perform the chroma subsampling using a rank filter and the second chroma subsampler could be configured to perform the chroma subsampling using an average filter such as a mean average filter. In accordance with the embodiments outlined below, the second chroma subsampler 16 performs its subsampling 26 using the chroma channels of image 18 only, i.e. irrespective of, and independent from, the luminance channel of image 18. The first chroma subsampler 14, however, may use a rank filter for chroma subsampling 24, which yields its ranking of the chroma samples in the original image 18 by evaluating the luminance channel of image 18. For example, the first chroma subsampler 14 kernel-wise selects a chroma sample value of the picture to be chroma subsampled, being co-located to a maximum or minimum luminance sample value of the image 18 to be chroma subsampled.
Next, specific embodiments or implementations are presented. They illustrate possible implementations of the detector, the first chroma subsampler 14 and the second chroma subsampler 16 but it is noted that all of these implementation details could be applied onto detector 12, chroma subsampler 14 and chroma subsampler 16, individually, without sticking to the combinations as they occur in the figures.
With regard to
In accordance with the embodiments of
For one exemplary local template position,
wherein i, j are within 1 . . . 4 in the case of
max({∀i≠j with i<j:|li−lj|})>t.
If the check reveals that the threshold is exceeded, the detector 12 attributes the area covered by the current local template position to the high contrast areas, i.e. to region 20, whereas if not the area covered by the current local template position is attributed to the remaining region 22. High contrast areas are, in
It should be noted that there are several modifications possible with respect to the description brought forward so far. For example, the high contrast area detection may be performed in a more complex manner. For example, the detector 12 may perform the high contrast area check at a pitch which, for example, is lower than the local templates width, i.e. the pitch at which the local template is positioned in order to cover image 18 and locally sample the contrast of image 18, may be lower than the local template's width, so that the templates overlap each other at the positions distributed over image 18 at the detection pitch. For example, detector 12 could perform the high contrast check for each of the 2×2 areas shown in
With respect to
However, in accordance with the embodiments of
However, it should be noted that even this example is merely for illustrative purposes. Instead of using the chroma sample value being co-located to the maximum luminance sample value of image 18, subsampler 14 could, for example, said y to be equal to the chroma sample value being co-located to the median of the luminance sample values within filter kernel 42, for example.
In order to illustrate further variations,
Summarizing the above embodiments, same used two different chroma subsampling methods with one being selected to avoid color bleeding artifacts and the other one being selected to avoid de-saturation artifacts. The color bleeding avoiding chroma subsampling method may, for example, use the chrominance values from pixel positions with highest luminance value in the 2×2 or 2×1 pixel areas as described above. A selection among the chroma subsampling methods may, as described above, be performed on the basis of the ratio of highest to lowest luminance value in 2×2 or 2×1 pixel areas. The pixel areas used in the chroma subsampling methods, and used in chroma subsampling method selection may coincide as described above. The ratio of the luminance values may be computed as a difference in case of using a logarithm domain to represent the luminance in image 18. In particular, the above described embodiments may be used to chroma subsample HDR images or video.
Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.
Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.
Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.
Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitionary.
A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.
A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.
In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are advantageously performed by any hardware apparatus.
While this invention has been described in terms of several embodiments, there are alterations, permutations, and equivalents which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and compositions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations and equivalents as fall within the true spirit and scope of the present invention.
Number | Date | Country | Kind |
---|---|---|---|
13167639 | May 2013 | EP | regional |
This application is a continuation of copending International Application No. PCT/EP2014/059382, filed May 7, 2014, which claims priority from European Application No. EP 13167639.7, filed May 14, 2013, which are each incorporated herein in its entirety by this reference thereto.
Number | Name | Date | Kind |
---|---|---|---|
5477345 | Tse | Dec 1995 | A |
5742704 | Koshi et al. | Apr 1998 | A |
7003160 | Horie et al. | Feb 2006 | B2 |
7084906 | Adams, Jr. | Aug 2006 | B2 |
20020025080 | Horie | Feb 2002 | A1 |
20030222998 | Yamauchi et al. | Dec 2003 | A1 |
20060104508 | Daly et al. | May 2006 | A1 |
20090278961 | Mohanty et al. | Nov 2009 | A1 |
20110052053 | Tomaselli | Mar 2011 | A1 |
20130064448 | Tomaselli | Mar 2013 | A1 |
20130322753 | Lim | Dec 2013 | A1 |
20160269733 | Tourapis | Sep 2016 | A1 |
Number | Date | Country |
---|---|---|
H04178094 | Jun 1992 | JP |
H06319133 | Nov 1994 | JP |
2002077625 | Mar 2002 | JP |
2002232293 | Aug 2002 | JP |
2004535126 | Nov 2004 | JP |
2006058253 | Mar 2006 | JP |
2011166547 | Aug 2011 | JP |
2011138056 | Mar 2013 | RU |
03007126 | Jan 2003 | WO |
2010104624 | Sep 2010 | WO |
2011088960 | Jul 2011 | WO |
2012142506 | Oct 2012 | WO |
Entry |
---|
Wong, Alexander, and William Bishop. “Practical content-adaptive subsampling for image and video compression.” Multimedia, 2006. ISM'06. Eighth IEEE International Symposium on. IEEE, 2006. |
Murching, Anil M., and John W. Woods. “Adaptive subsampling of color images.” Image Processing, 1994. Proceedings. ICIP—94., IEEE International Conference. vol. 3. IEEE, 1994. |
Sullivan, Gary , “Color Format Down-Conversion for Test Sequence Generation”, Dec. 2003, pp. 1-10. |
Sullivan, Gary , “Color Format Up-Conversion for Video Display”, Dec. 2003, pp. 1-6. |
Wikipedia, “JPEG”, Retrieved from the Internet: URL:http://en.wikipedia.org/w/index.php?title=JPEG&oldid=554886142[retrieved on Nov. 1, 2013], May 13, 2013, 20 pages. |
Number | Date | Country | |
---|---|---|---|
20160071251 A1 | Mar 2016 | US |
Number | Date | Country | |
---|---|---|---|
Parent | PCT/EP2014/059382 | May 2014 | US |
Child | 14941056 | US |