Patent Application
20190014235
The application claims priority from European Patent Application No. 17305871.0, entitled “METHOD AND DEVICE FOR ADAPTING A DISPLAY VISIBILITY”, filed on Jul. 6, 2017, the contents of which are hereby incorporated by reference in its entirety.
The present disclosure relates to the domain of image display in varying viewing conditions.
Visible display of images is challenging in bright conditions, as for example in a sunny weather outdoor environment. Some known methods propose to increase the luminance of display screens to improve the visibility of a displayed image under very bright conditions. Although these techniques improve the visibility of high quality images, they do not necessarily perform well for lower quality images. In case images comprise for example noise or blurred areas, increasing the luminance however does not improve the overall visibility of the image. There is a need for new methods for improving a visibility of a wider range of images in varying illuminating conditions.
In a context of varying ambient luminosity, a salient idea is to adjust an image by adapting the contrast of an image according to a level of ambient light. Enhancing the contrast is known to make a displayed object more distinguishable, with some impact on the overall image quality. Increasing a level of contrast enhancement as the ambient light increases advantageously allows to preserve distinguishable displayed objects in high levels of ambient lights, despite an overall loss of quality, less perceivable in high levels of ambient light.
To that end a method for adapting a display visibility of an image is disclosed. The method comprises:
According to a variant, the image adjustment is performed or not performed as a function of the level of ambient luminosity.
According to another variant, a signal representing the image is separated into a high frequency portion and a low frequency portion, and adjusting the image comprises modifying the low frequency portion of the signal representing the image by applying a transfer function to the low frequency portion, the transfer function depending on the level of ambient luminosity, the modified low frequency portion being further combined with the high frequency portion of the signal representing the image before being provided for display.
According to another variant, the high frequency portion is amplified with a coefficient depending on the level of ambient luminosity, prior to be combined with the modified low frequency portion.
According to another variant, a parameter of the transfer function, obtained by a first function increasing with the level of ambient luminosity.
According to another variant, the coefficient is obtained by a second function increasing with the level of ambient luminosity.
According to another variant, the coefficient is obtained by a fourth function decreasing with a perceived contrast level, the perceived contrast level representing a combination of a contrast level of the image and the level of ambient luminosity.
According to another variant, the perceived contrast level is determined by locally adjusting the contrast level with a third function decreasing with the level of ambient luminosity
In a second aspect, a display device for adapting a display visibility of an image is also disclosed. The device comprises:
In a third aspect a display device for adapting a display visibility of an image is also disclosed. The device comprises:
In a fourth aspect, a computer program product for adapting a display visibility of an image is also disclosed. The computer program product comprises program code instructions executable by a processor for performing the method implemented in any of its variant.
In a fifth aspect, a non-transitory computer-readable storage medium storing computer-executable program instructions for adapting a display visibility of an image is also disclosed. The computer-readable storage medium comprises instructions of program code executable by at least one processor to perform the method implemented in any of its variant.
While not explicitly described, the present embodiments may be employed in any combination or sub-combination. For example, the present principles are not limited to the described variants, and any arrangement of variants and embodiments can be used. Moreover, the present principles are not limited to the described parametric contrast enhancement techniques examples and any other type of parametric contrast enhancement is compatible with the disclosed principles. The present principles are not further limited to the described continuously increasing or decreasing function and are applicable to any other continuously increasing or decreasing function. The present principles are not further limited to the described sharpening technique.
Besides, any characteristic, variant or embodiment described for a method is compatible with a device comprising means for processing the disclosed method, with a device comprising a processor configured to process the disclosed method, with a computer program product comprising program code instructions and with a non-transitory computer-readable storage medium storing program instructions.
It should be understood that the drawing(s) are for purposes of illustrating the concepts of the disclosure and are not necessarily the only possible configuration for illustrating the disclosure.
It should be understood that the elements shown in the figures may be implemented in various forms of hardware, software or combinations thereof. Preferably, these elements are implemented in a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, memory and input/output interfaces. Herein, the phrase “coupled” is defined to mean directly connected to or indirectly connected with through one or more intermediate components. Such intermediate components may include both hardware and software based components.
The present description illustrates the principles of the present disclosure. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the disclosure and are included within its scope.
All examples and conditional language recited herein are intended for educational purposes to aid the reader in understanding the principles of the disclosure and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions.
Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.
Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative circuitry embodying the principles of the disclosure. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudocode, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, read only memory (ROM) for storing software, random access memory (RAM), and nonvolatile storage.
Other hardware, conventional and/or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.
In the claims hereof, any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function. The disclosure as defined by such claims resides in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.
The present disclosure addresses issues related to display visibility adaptation while displaying an image in a varying illuminated environment. The present principles are applicable to images of a video sequence.
In the step S12, an adjusted image is obtained by adapting the contrast of the image according to the measured level L(t) of ambient luminosity. In an advantageous variant, the image adjustment is performed or not performed as a function of the level of ambient luminosity (performed for high values of ambient luminosity and not performed for low values of ambient luminosity). In a non-limiting example, the contrast of the image is adapted only in case the measured level of ambient luminosity is above a given value. Indeed, in low light conditions, such as indoor or night environment, the viewing conditions are considered as good for the users, and adapting the contrast would create a risk to generate visually unpleasant and/or tiring effects. Moreover, adapting the contrast is also likely to impact the power consumption of the device and to be damageable for the battery duration. A given value is for example a value between 100 lux and 300 lux. The given value is for example a configuration parameter, adjustable by the user via a user interface. In another example, the given value is a parameter configured by the display device manufacturer. In yet another example, the given value is automatically determined by the display device according to a user profiling technique.
According to a non-limiting embodiment, the contrast is adapted by enhancing the contrast according to the measured level L(t) of ambient light. More precisely the contrast is enhanced on the adjusted image by applying a contrast enhancement technique of any type, parametrized by a contrast enhancement parameter μ which depends on the level of ambient luminosity.
In other words, the luminance of the adjusted image is obtained by applying a parametric transfer function, noted “CEμ” to the luminance of the image depending on the currently measured level of ambient luminosity. The notation “CEμ” reflects that the transfer function, noted “CE” is a parametric transfer function depending on the parameter μ, which itself depends on the ambient luminosity. As the level of measured ambient luminosity is varying over time, the contrast enhancement parameter μ and thus the parametric transfer function also vary over time as the level of ambient luminosity vary. For example, the parametric transfer function CEμ is parametrized by the contrast enhancement parameter μ. In a first variant, the contrast enhancement is a linear stretching, and the transfer function is linear. The μ parameter is the shape of the transfer function. In a second variant, the contrast enhancement is a piecewise linear stretching, and the transfer function is a piece wise linear transfer function comprising successive linear curbs with different shapes. According to the second variant, the μ parameter is related to the highest shape among the different shapes of the different linear pieces of the of the piece wise linear transfer function. In yet another variant, the contrast enhancement comprises a gamma transform. In this case the μ parameter is related to the gamma. Any other variant of parametric contrast enhancement function, for which the parameter depends of the level of ambient luminosity is compatible with the disclosed principles. Advantageously, a parameter of the transfer function (the contrast enhancement parameter μ) is obtained by a first function, noted μ=g(L(t)), where “t” represent the time and L(t) a level of ambient luminosity at time t. The first function g(L(t)) is strictly and continuously increasing with the level luminosity. Any parametric type of strictly and continuously increasing function including but not limited to linear, logarithmic and polynomial are compatible with the disclosed principles. Different parametric types of strictly and continuously increasing functions differently impact the a same variation of ambient light because of their different shaping (linear, log, polynomial). Advantageously, the parametric type of increasing function is determined according to the display screen characteristics, and, is for example performed through user tests.
In the step S14, the display visibility of the image is adapted by sending the adjusted image for display on a display device.
According to a first optional and non-limiting embodiment, the image (noted I) is separated into a high-frequency component Ih and a low frequency component Il, which is equal to I−Ih. In other words, a signal represented the image is separated into a low frequency part or portion and a high frequency part of portion. The separation may be performed by filtering the signal using many different types of filters including, but not limited to, an iterated median filter, edge preserving filters, bilateral filter, and a rolling guidance filter. Advantageously, in this optional variant, only the low frequency component Il is modified by applying the transfer function depending on the level of ambient luminosity (as described above), the modified low frequency component being further recombined with the high frequency component for obtaining the adjusted image. Enhancing the contrast only on the low frequency component is advantageous as the low frequency component has a higher signal dynamic than the high frequency component. Typically, in the high frequency component, an edge is spread only on a few values, and further spreading these few values (by applying the contrast enhancement to them) are more likely to create undesired artefacts. Without loss of generality, recombining the two components corresponds to adding both components, representing the inverse operation of the separation. More formally, the adjusted image Ia according to the second optional embodiment can be written as equation (1):
I
a
=CE
μ*(I−Ih); μ=g(L(t) (1)
According to a second optional and non-limiting embodiment, the image is also separated into a high Ih and a low Il frequency components as for the first embodiment. In other words, a signal represented the image is separated into a low frequency part or portion and a high frequency part of portion, and the high frequency portion of the signal (component Ih of the image) is amplified to enhance edges and sharpen the image prior being recombined with the modified low frequency portion of the signal (component of the image as described in the first embodiment). More precisely, adjusting the image further comprises amplifying the high frequency portion of the signal (component Ih) of the image with an amplification coefficient a depending on the measured level L(t) of ambient luminosity at time t. More formally, the adjusted image Ia according to the second optional embodiment can be written as equation (2):
Ia=CEμ*(I−Ih)+α*Ih (2)
Global Sharpening
In a first variant of the second optional embodiment (called global sharpening), the amplification coefficient a is determined globally for the image by a second function of the measured level L(t) of ambient luminosity (α=g′(L(t))), the second function g′ increasing with the measured level of ambient luminosity. The second function may be of any parametric type including but not limited to linear, polynomial, sine and exponential functions. In a first example, the first and the second functions are a same function. In another example, they are different functions. The type of function depends on the display characteristic such as its gamut or its sensitivity, and is advantageously tuned according to user tests. In global sharpening variant, the amplification coefficient is a global amplification coefficient and does not spatially vary: the global coefficient has the same value for all the pixels of the image, and only depends on the measured level L(t) of ambient luminosity.
Local Sharpening
In a second variant of the second optional embodiment (called local sharpening), the amplification coefficient a is determined locally in the image depending on both the color values of elements of the image and the measured level of ambient light. The term “element” refers to any part of the image, associated with given color characteristics represented for example by at least three color component values. For example, an element corresponds to a pixel in an image. In another example, the element corresponds to a set of pixels in an image with similar color component values. In another example, an element corresponds to a patch on which the contrast has been extracted. For the sake of clarity and without limitation, an element is considered having spatial coordinates (x,y). Depending on what an element represents, the spatial coordinates are for example integers (representing a pixel coordinates), or sets of integers (representing for example sets of pixels). Considering Ia(x,y) representing the adjusted image value at each point/element (x,y), equation (2) is rewritten as:
I
a(x,y)=CEμ*(I(x,y)−Ih(x,y))+α(x,y)*Ih(x,y) (2)
For the sake of clarity, the values Ia(x,y),(I(x,y) and Ih(x,y) represent a component value of an area corresponding to the spatial coordinates (x,y) in the corresponding image. Without loss of generality a component value may be a luminance value or a triplet of Red, Green, Blue colour component values. According to the local sharpening variant, the amplification coefficient is a local amplification coefficient, varying locally in the image depending on the local colour value of the image and further depending on the measured level of light. The contrast is known as the difference in luminance or in colour that makes an object (or its representation in an image) distinguishable. In visual perception of the real world, contrast is generally determined by the difference in the colour and the brightness of the object and other objects with a same field of view. Advantageously, a perceived contrast is determined for the image, the perceived contrast being defined as a mixture between the contrast of the image and the measured level of ambient light, the higher the level of ambient light, the lesser the perceived contrast. The perceived contrast, contrary to the contrast is a metric representing how objects are distinguishable under a given ambient lighting condition.
Determining a perceived contrast Pt(x,y) of an image I(x,y) comprises extracting a contrast Ct(x,y) of the image. For example, a root mean square (RMS) contrast is extracted. In a nutshell, it amounts to compute the standard deviation of the luminance signal over patches (rectangles) of a given size. Depending on the size (and the number) of patches the accuracy of the contrast and the required computational resources for extracting it vary. Any other contrast metric including but not limited to the Weber contrast or the Michelson contrast and their corresponding extraction techniques are compatible with the disclosed principles. Computing the RMS contrast is advantageous as it is not computational intensive and is compatible with the computing resources available on mobile devices such as smartphones or tablets.
Determining the perceived contrast Pt(x,y) of the image I(x,y) further comprises locally adjusting the extracted contrast level with a third function decreasing with the measured level of luminosity. For example, the extracted local contrast level Ct(x,y) is multiplied by the third function comprising an exponential function and a scale parameter σ, according to the following formula:
P
t(x,y)=Ct(x,y)*exp(−L(t)/2σ2) (3)
The scale parameter σ determines the cut between the level of ambient luminosity and the perceived contrast. It depends on the display device characteristic and is advantageously tuned through user tests.
According to the local sharpening variant, the local coefficient α(x,y) of an element in the image is obtained by a fourth function f of the perceived contrast level Pt(x,y) of the element in the image, the fourth function f decreasing with the perceived contrast level Pt. (α(x,y)=f(Pt(x,y))). As for the first embodiment, the fourth function may be of any parametric type including but not limited to linear, polynomial, cosine and exponential functions.
Unsharp masking is an image sharpening technique. The “unsharp” of the name derives from the fact that the technique uses a blurred (or unsharp), negative image to create a mask of the original image. The unsharp mask is then combined with the positive (original) image, creating an image that is less blurry than the original. The resulting image, although clearer is generally a less accurate representation of the image's subject. The unsharp mask is generally a linear or nonlinear filter that amplifies the high-frequency component of an image. The local and the global sharpening variants advantageously adapt the unsharp masking technique by determining a perceived contrast level according to a level of ambient light, and amplifying the high frequency component of an image depending on the perceived contrast to increase the level of sharpening as the level of ambient light increases.
The local sharpening variant is further advantageous, as the sharpening is concentrated on the most contrasted areas, representing visually meaningful area, letting other more homogenous areas unchanged (or less sharpened). The local sharpening variant limits the drawback of strong sharpening in homogenous areas (creating noise amplification and/or visually unpleasant effects).
According to any variant described above the adjusted image, corresponding to the sharpened image is provided for display on the display device.
In an advantageous variant, the operations of the combinations or recombinations (addition/subtraction and multiplication) on images are performed in a Generalized Linear System (GLS) as proposed in “A generalized unsharp masking algorithm” by Deng (in IEEE Transactions on Image Processing, 2011) so as to remain in the coding domain of the image.
The processing device 3 further comprises an optional input 31 to receive configuration data from a user. Configuration data are generated by a user via a user interface in order to configure the processing device 3. According to different embodiments of the disclosed principles, the user interface belongs to a set comprising:
More generally any user interface allowing to provide configuration data is compatible with disclosed principles.
The processing device 3 further comprises a sensor 32, for example a light detector, configured to receive and measure a level of ambient luminosity from an ambient environment. The sensor if for example the light detector of the camera embedded in a smartphone. Any light detector capable to detect and measure an amount of ambient luminosity is compatible with the disclosed principles.
The inputs 30 and 31 and the light detector 32 are linked to a processing module 34 configured to adjust the image by adapting the contrast of the image according to the level of ambient luminosity. The processing module 34 is further configured to adapt the display visibility of the image by sending the adjusted image to a display mean 38.
According to different embodiments of the disclosed principles, the display mean 38 belongs to a set comprising:
More generally any display mean allowing to display an adjusted image, is compatible with the disclosed principles.
According to an exemplary and non-limiting embodiment, the processing device 3 further comprises a computer program stored in the memory 420. The computer program comprises instructions which, when executed by the processing device 3, in particular by the processor 410, make the processing device 3 carry out the processing method described with reference to
According to exemplary and non-limiting embodiments, the processing device 3 is a display device to be used in a bright environment (possibly outdoor but not limited to out-door environments), which belongs to a set comprising:
| Number | Date | Country | Kind |
|---|---|---|---|
| 17305871.0 | Jul 2017 | EP | regional |
