This relates generally to electronic devices with displays and, more particularly, to electronic devices with displays that adapt to different ambient lighting conditions.
The chromatic adaptation function of the human visual system allows humans to generally maintain constant perceived color under different ambient lighting conditions. For example, white paper will appear white to the human eye even when illuminated under different ambient lighting conditions.
Conventional displays do not typically account for different ambient lighting conditions or the chromatic adaptation of the human visual system. As a result, a user may perceive undesirable color shifts in the display under different ambient lighting conditions. For example, the white point of a display may appear white to a user in outdoor ambient lighting conditions, but may appear bluish to the user in an indoor environment when the user's eyes have adapted to the warmer light produced by indoor light sources. Similarly, white light emitted from the display under a cool white light source may appear red to a viewer who has adapted to the cool white light.
It would therefore be desirable to be able to provide improved ways of displaying images with displays.
An electronic device may include a display having an array of display pixels and having display control circuitry that controls the operation of the display. The display control circuitry may adaptively adjust the output from the display based on ambient lighting conditions.
The display control circuitry may operate the display in different modes depending on the ambient lighting conditions. For example, the electronic device may include a color-sensitive light sensor that measures the brightness and color of ambient light. Display control circuitry may determine which mode to operate the display in based on the ambient light sensor data.
In a paper mode, display control circuitry may use stored spectral reflectance data (e.g., spectral reflectance data that describes the reflectance spectra of colors printed on paper) to adjust display colors such that the colors appear as they would on a printed sheet of paper. This may include, for example, adjusting pixel data based on the spectral reflectance data associated with the color to be produced as well as the color and intensity of ambient light measured by the color-sensitive light sensor. The adjusted pixel data may be provided to the pixel array to produce the desired color.
In a low light mode when the ambient light level is below a threshold, the light emitted from the display may be adjusted to mimic the appearance of an incandescent light source. In a bright light mode when the ambient light level exceeds a threshold, the light emitted from the display may be adjusted to maximize readability in bright light. The target white point of the display may be selected depending on which mode the display is operating in. In low light mode, for example, the target white point may be shifted towards the yellow portion of the spectrum to produce warm white light, which may in turn have beneficial effects on the human circadian rhythm by displaying warmer colors in the evening.
Further features of the invention, its nature and various advantages will be more apparent from the accompanying drawings and the following detailed description of the preferred embodiments.
Electronic devices such as cellular telephones, media players, computers, set-top boxes, wireless access points, and other electronic equipment may include displays. Displays may be used to present visual information and status data and/or may be used to gather user input data.
An illustrative electronic device of the type that may be provided with an ambient light adaptive display is shown in
As shown in
Device 10 may have a housing such as housing 12. Housing 12, which may sometimes be referred to as a case, may be formed of plastic, glass, ceramics, fiber composites, metal (e.g., stainless steel, aluminum, etc.), other suitable materials, or a combination of any two or more of these materials.
Housing 12 may be formed using a unibody configuration in which some or all of housing 12 is machined or molded as a single structure or may be formed using multiple structures (e.g., an internal frame structure, one or more structures that form exterior housing surfaces, etc.).
As shown in
In the example of
A schematic diagram of device 10 is shown in
With one suitable arrangement, storage and processing circuitry 40 may be used to run software on device 10 such as internet browsing applications, email applications, media playback applications, operating system functions, software for capturing and processing images, software implementing functions associated with gathering and processing sensor data, software that makes adjustments to display brightness and touch sensor functionality, etc.
To support interactions with external equipment, storage and processing circuitry 40 may be used in implementing communications protocols. Communications protocols that may be implemented using storage and processing circuitry 40 include internet protocols, wireless local area network protocols (e.g., IEEE 802.11 protocols—sometimes referred to as WiFi®), protocols for other short-range wireless communications links such as the Bluetooth® protocol, etc.
Input-output circuitry 32 may be used to allow input to be supplied to device 10 from a user or external devices and to allow output to be provided from device 10 to the user or external devices.
Input-output circuitry 32 may include wired and wireless communications circuitry 34. Communications circuitry 34 may include radio-frequency (RF) transceiver circuitry formed from one or more integrated circuits, power amplifier circuitry, low-noise input amplifiers, passive RF components, one or more antennas, and other circuitry for handling RF wireless signals. Wireless signals can also be sent using light (e.g., using infrared communications).
Input-output circuitry 32 may include input-output devices 36 such as button 22 of
Sensor circuitry such as sensors 38 of
During operation of device 10, storage and processing circuitry 40 may produce data that is to be displayed on display 14. This display data may be provided to display control circuitry such as timing controller integrated circuit 126 using graphics processing unit 124.
Timing controller 126 may provide digital display data to column driver circuitry 120 using paths 128. Column driver circuitry 120 may receive the digital display data from timing controller 126. Using digital-to-analog converter circuitry within column driver circuitry 120, column driver circuitry 120 may provide corresponding analog output signals on the data lines D running along the columns of display pixels 52 of array 92.
Storage and processing circuitry 40, graphics processing unit 124, and timing controller 126 may sometimes collectively be referred to herein as display control circuitry 30. Display control circuitry 30 may be used in controlling the operation of display 14.
Each pixel 52 may, if desired, be a color pixel such as a red (R) pixel, a green (G) pixel, a blue (B) pixel, a white (W) pixel, or a pixel of another color. Color pixels may include color filter elements that transmit light of particular colors or color pixels may be formed from emissive elements that emit light of a given color.
Pixels 52 may include pixels of any suitable color. For example, pixels 52 may include a pattern of cyan, magenta, and yellow pixels, or may include any other suitable pattern of colors. Arrangements in which pixels 52 include a pattern of red, green, and blue pixels are sometimes described herein as an example.
Display control circuitry 30 and associated thin-film transistor circuitry associated with display 14 may be used to produce signals such as data signals and gate line signals for operating pixels 52 (e.g., turning pixels 52 on and off, adjusting the intensity of pixels 52, etc.). During operation, display control circuitry 30 may control the values of the data signals and gate signals to control the light intensity associated with each of the display pixels and to thereby display images on display 14.
Display control circuitry 30 may obtain red, green, and blue pixel values (sometimes referred to as RGB values or digital display control values) corresponding to the color to be displayed by a given pixel. The RGB values may be converted into analog display signals for controlling the brightness of each pixel. The RGB values (e.g., integers with values ranging from 0 to 255) may correspond to the desired pixel intensity of each pixel. For example, a digital display control value of 0 may result in an “off” pixel, whereas a digital display control value of 255 may result in a pixel operating at a maximum available power.
It should be appreciated that these are examples in which eight bits are dedicated to each color channel. Alternative embodiments may employ greater or fewer bits per color channel. For example, if desired, six bits may be dedicated to each color channel. With this type of configuration, RGB values may be a set of integers ranging from 0 to 64. Arrangements in which each color channel has eight bits dedicated to it are sometimes described herein as an example.
As shown in
Light sensors such as color light sensors 42 and cameras may, if desired, be distributed at different locations on electronic device 10 to detect light from different directions. Other sensors such as an accelerometer and/or gyroscope may be used to determine how to weight the sensor data from the different light sensors. For example, if the gyroscope sensor data indicates that electronic device 10 is placed flat on a table with display 14 facing up, electronic device 10 may determine that light sensor data gathered by rear light sensors (e.g., on a back surface of electronic device 10) should not be used.
Display control circuitry 30 may be configured to adaptively adjust the output from display 14 based on ambient lighting conditions. In adjusting the output from display 14, display control circuitry 30 may take into account the chromatic adaptation function of the human visual system. This may include, for example, determining characteristics of the light that the user's eyes are exposed to.
To avoid the perceived discoloration of display 14, display control circuitry 30 of
Display control circuitry 30 may, if desired, adjust the color and brightness of light emitted from display 14 to mimic the appearance of a diffusely reflective object illuminated only by surrounding ambient light. In some scenarios, display 14 may be indistinguishable from a printed sheet of paper.
When viewing an object in ambient light, the spectrum of light that reaches one's eye is a function of the surrounding illuminants and the object's reflectivity spectrum. Thus, to mimic the appearance of a diffusely reflective object illuminated by ambient light, display control circuitry 30 may determine the brightness and color of ambient light using color-sensitive light sensor 42 (
In some ambient lighting conditions, it may not be desirable to mimic the appearance of a diffusely reflective object. For example, in low light levels where the display light is the main source of illumination around a user, it may be desirable to mimic the appearance of an indoor light source. In bright lighting conditions, it may be desirable to maximize readability.
To address these different scenarios, display control circuitry 30 may operate display 14 in different modes depending on the ambient lighting conditions. In a given display mode, display control circuitry 30 may adjust display light to achieve a given result.
The intensity of ambient light incident on display 14 may be measured by a light sensor in electronic device 10 such as color-sensitive light sensor 42 of
In one suitable arrangement, which is sometimes described herein as an illustrative example, display control circuitry 30 may operate display 14 in a “low light mode” when light sensor 42 indicates ambient light levels are between L0 and L1, a “paper mode” when light sensor 42 indicates ambient light levels are between L1 and L2, and a “bright light mode” when light sensor 42 indicates ambient light levels are greater than L2.
L1 may be about 8.4 lux, about 8.5 lux, about 8.0 lux, greater than 8.0 lux, or less than 8.0 lux. L2 may be about 850 lux, about 900 lux, about 800 lux, greater than 800 lux, or less than 800 lux.
In paper mode, display control circuitry 30 may adjust display light such that the appearance of displayed images mimics that of a diffusely reflective object such as paper. This may include, for example, determining the brightness and color of ambient light using color-sensitive light sensor 42 and then using known reflectivity behavior of the colors that the display is attempting to reproduce to adjust the color and brightness of display light such that the displayed images mimic the appearance of diffusely reflective objects. As shown in
For most ambient lighting conditions (e.g., between illuminance values L1 and L2), operating display 14 to mimic the appearance of printed paper may be the desirable mode of operation. In dim lighting conditions or very bright lighting conditions, however, it may be desirable to achieve other effects with display 14. To account for these different ambient lighting conditions, display control circuitry 30 may operate display 14 in low light mode when the ambient light levels are less than L1 and in bright light mode when ambient light levels are greater than L2.
In low light mode, it may not be desirable to mimic the appearance of printed paper because the ambient light may be too dim to sufficiently illuminate the displayed images. For example, when ambient light levels fall below L1, the luminance of paper may approach D0. If display 14 were also to approach D0 in dim ambient light, a user may find it difficult to read text or see images on display 14. Rather, since the light emitted from display 14 is the primary source of illumination in the vicinity of the user and there is no external source of illumination to adapt to, display control circuitry 30 may transition display 14 into self-illuminating low light mode (sometimes referred to as “lamp mode”). In low light mode, the white point of display 14 may be set to any desired white point, and display luminance levels may be kept at or above a desired minimum such as D1. D1 may, for example, be about 2.4 nits, about 2.5 nits, about 3.0 nits, greater than 3.0 nits, or less than 3.0 nits.
The white point of a display is commonly defined by a set of chromaticity coordinates that represent the color produced by the display when the display is generating all available display colors at full power. Prior to any corrections during calibration, the white point of the display may be referred to as the “native white point” of that display. Due to manufacturing differences between displays, the native white point of a display may differ, prior to calibration of the display, from the desired (target) white point of the display. The target white point may be defined by a set of chromaticity values associated with a reference white (e.g., a white produced by a standard display, a white associated with a standard illuminant such as the D65 illuminant of the International Commission on Illumination (CIE), a white produced at the center of a display). In general, any suitable white point may be used as a target white point for a display.
Using the display modes of
If desired, the low light mode white point may be adjusted to achieve beneficial effects on the human circadian rhythm. The human circadian system may respond differently to different wavelengths of light. For example, when a user is exposed to blue light having a peak wavelength within a particular range, the user's circadian system may be activated and melatonin production may be suppressed. On the other hand, when a user is exposed to light outside of this range of wavelengths or when blue light is suppressed (e.g., compared to red light), the user's melatonin production may be increased, signaling nighttime to the body.
Conventional displays do not take into account the spectral sensitivity of the human circadian rhythm. For example, some displays emit light having spectral characteristics that trigger the circadian system regardless of the time of day, which can in turn have an adverse effect on sleep quality.
In contrast, by operating the display in low light mode when the ambient light falls below level L1 (e.g., at night when a user is indoors), the neutral point of display 14 may become warmer (e.g., may tend to the yellow portion of the spectrum) in dim ambient lighting conditions. Thus, when a user is at home in the evening (e.g., reading in warm ambient light), blue light emitted from display 14 may be suppressed as the display adapts to the ambient lighting conditions. The reduction in blue light may in turn reduce suppression of the user's melatonin production (or, in some scenarios, may increase the user's melatonin production) to promote better sleep.
This is, however, merely illustrative. In general, the white point of display 14 and the characteristics of neutral colors displayed by display 14 may be adjusted in any desirable fashion in low light mode. Since the ambient light from external light sources is not sufficiently bright to have a significant effect on the chromatic adaptation of the user's vision, the color and brightness of display 14 may be adjusted freely (e.g., based on user preferences, based on the time of day, etc.). As shown in
In bright ambient light (e.g., outdoors, in direct sunlight, etc.), it may also be desirable to change the mode of operation of display 14 from paper mode to a different mode of operation. For example, in ambient light levels above L2, the luminance of paper may exceed D2, but it may not be desirable or practical to exceed luminance D2 with display 14 to match the appearance of paper. Rather, display control circuitry 30 may operate display 14 to maximize readability by increasing brightness and contrast of displayed images. In some scenarios, this may include operating display 14 at luminance levels at or below D2 when ambient light levels exceed L2. D2 may be about 240 nits, about 250 nits, about 230 nits, less than 230 nits, or greater than 230 nits.
At step 300, display control circuitry 30 may receive incoming pixel values indicating display colors to be displayed by display 14. This may include, for example, receiving a frame of display data including red, green, and blue pixel values (sometimes referred to as RGB values or digital display control values) corresponding to the color to be displayed by a pixel in the frame of display data.
At step 302, display control circuitry 30 may gather light information from one or more light sensors such as color-sensitive light sensor 42 of
At step 304, display control circuitry 30 may determine a display mode based on the brightness of the ambient light. When ambient light levels are below a threshold brightness (e.g., below illuminance value L1 of
At step 306, display control circuitry 30 may operate display 14 in low light mode. In low light mode, the light emitted from display 14 is the primary source of illumination in the vicinity of the user and there is no external source of illumination to adapt to. Step 306 may include adjusting the chromaticity values associated with the target white point for display 14. In low light mode, the target white point of display 14 may be set to any desired white point, and display luminance levels may be kept at or above a desired minimum (e.g., above luminance value D1 of
If desired, the low light mode white point may be adjusted to achieve beneficial effects on the human circadian rhythm. This may include, for example, adjusting the neutral point of display 14 to be warmer (e.g., may tend to the yellow portion of the spectrum) in dim ambient lighting conditions. The neutral point in low light mode may be adjusted so that the light emitted from display 14 matches the color and brightness characteristics of a typical indoor light source (e.g., to mimic the appearance of an incandescent light bulb or other desired light source). Thus, when a user is at home in the evening (e.g., reading in warm ambient light), blue light emitted from display 14 may be suppressed as the display adapts to the ambient lighting conditions. The reduction in blue light may in turn reduce suppression of the user's melatonin production (or, in some scenarios, may increase the user's melatonin production) to promote better sleep.
This is, however, merely illustrative. In general, the white point of display 14 and the characteristics of neutral colors displayed by display 14 may be adjusted in any desirable fashion in low light mode. Since the ambient light from external light sources is not sufficiently bright to have a significant effect on the chromatic adaptation of the user's vision, the color and brightness of display 14 may be adjusted freely (e.g., based on user preferences, based on the time of day, etc.) to achieve the desired lighting effect.
If it is determined in step 304 that the ambient light level is within a given range of values (e.g., between illuminance values L1 and L2 of
At step 308, display control circuitry 30 may adjust display light to mimic the appearance of printed paper. Since the way a user perceives a diffusely reflective object depends on the color and brightness of ambient light and the object's spectral reflectance, display control circuitry 30 may adjust display light based on the ambient light brightness and color information gathered in step 302 and based on the known reflectivity behavior of the colors that display 14 is intended to reproduce (e.g., based on the pixel data received in step 300 and based on stored spectral reflectance data).
Reflectivity information indicating reflectivity behavior of different colors may be stored in electronic device 10 (e.g., in storage and processing circuitry 40) and may be used to determine how display light should be adjusted in step 308. For example, light reflected off of a red image on a printed piece of paper may have first color characteristics under a first type of illuminant and second color characteristics under a second type of illuminant. Using this type of spectral reflectance information, display control circuitry 30 may determine how to adjust display colors to mimic that of a diffusely reflective object under a given illuminant. This may include, for example, using a first set of RGB pixel values to display a given image under a first illuminant, and a second set of RGB pixel values to display the same image under a second illuminant. The first and second illuminants may have the same intensity but may have slightly different color characteristics, which would be detected by sensor 42 and accounted for in step 308.
If it is determined in step 304 that the ambient light level exceeds a given threshold (e.g., illuminance value L2 of
At step 310, display control circuitry 30 may adjust display light to maximize readability by increasing the contrast and brightness of images on display 14.
The foregoing is merely illustrative of the principles of this invention and various modifications can be made by those skilled in the art without departing from the scope and spirit of the invention. The foregoing embodiments may be implemented individually or in any combination.
This application is a continuation of U.S. patent application Ser. No. 15/388,416, filed Dec. 22, 2016, which is a continuation of U.S. patent application Ser. No. 14/673,667, filed Mar. 30, 2015, now U.S. Pat. No. 9,530,362, which claims priority to U.S. provisional patent application No. 62/096,188, filed Dec. 23, 2014, all of which are hereby incorporated by reference herein in their entireties.
Number | Name | Date | Kind |
---|---|---|---|
5710876 | Peercy et al. | Jan 1998 | A |
5726672 | Hernandez et al. | Mar 1998 | A |
5952992 | Helms | Sep 1999 | A |
6459436 | Kumada et al. | Oct 2002 | B1 |
6480202 | Deguchi et al. | Nov 2002 | B1 |
6611297 | Akashi et al. | Aug 2003 | B1 |
6628822 | Nakabayashi et al. | Sep 2003 | B1 |
6870567 | Funston et al. | Mar 2005 | B2 |
6947017 | Gettemy | Sep 2005 | B1 |
6952195 | Arakawa | Oct 2005 | B2 |
7142218 | Yoshida et al. | Nov 2006 | B2 |
7468722 | Ferguson | Dec 2008 | B2 |
7969478 | Chen et al. | Jun 2011 | B1 |
7973779 | Masalkar et al. | Jul 2011 | B2 |
8274718 | Kawai et al. | Sep 2012 | B2 |
8350787 | Tanba et al. | Jan 2013 | B2 |
8520022 | Cohen et al. | Aug 2013 | B1 |
8848003 | Takahashi et al. | Sep 2014 | B2 |
9965999 | Barnes | May 2018 | B1 |
20010040588 | Shiraiwa et al. | Nov 2001 | A1 |
20010050757 | Yoshida et al. | Dec 2001 | A1 |
20020080245 | Parulski et al. | Jun 2002 | A1 |
20020118294 | Takano et al. | Aug 2002 | A1 |
20020158812 | Pallakoff | Oct 2002 | A1 |
20040070565 | Nayar et al. | Apr 2004 | A1 |
20050280869 | Kameyama | Dec 2005 | A1 |
20060007223 | Parker | Jan 2006 | A1 |
20060109218 | Kim et al. | May 2006 | A1 |
20070097333 | Zavarehi et al. | May 2007 | A1 |
20070126727 | Chiang | Jun 2007 | A1 |
20080142687 | Zhou et al. | Jun 2008 | A1 |
20080165203 | Panffoerder | Jul 2008 | A1 |
20080180426 | Liu et al. | Jul 2008 | A1 |
20080204384 | Lee | Aug 2008 | A1 |
20080279451 | Shimbaru | Nov 2008 | A1 |
20080303918 | Keithley | Dec 2008 | A1 |
20090040205 | Scheibe | Feb 2009 | A1 |
20090096745 | Sprague et al. | Apr 2009 | A1 |
20090109129 | Cheong et al. | Apr 2009 | A1 |
20090153888 | Edge | Jun 2009 | A1 |
20090167950 | Chen et al. | Jul 2009 | A1 |
20090195670 | Koishi | Aug 2009 | A1 |
20090201309 | Demos | Aug 2009 | A1 |
20100060674 | Yoshida et al. | Mar 2010 | A1 |
20100060911 | Marcu et al. | Mar 2010 | A1 |
20100194289 | Katayanagi et al. | Aug 2010 | A1 |
20100320919 | Gough et al. | Dec 2010 | A1 |
20110206348 | Numakami | Aug 2011 | A1 |
20110249141 | Chen | Oct 2011 | A1 |
20120019493 | Barnhoefer et al. | Jan 2012 | A1 |
20120050307 | Mahowald | Mar 2012 | A1 |
20120081279 | Greenebaum et al. | Apr 2012 | A1 |
20120081663 | Schmid | Apr 2012 | A1 |
20120182276 | Kee | Jul 2012 | A1 |
20120182278 | Ballestad | Jul 2012 | A1 |
20120268437 | Lee | Oct 2012 | A1 |
20120287605 | Chen et al. | Nov 2012 | A1 |
20130093783 | Sullivan et al. | Apr 2013 | A1 |
20130328842 | Barnhoefer | Dec 2013 | A1 |
20140002428 | Letourneur et al. | Jan 2014 | A1 |
20140028197 | van de Ven | Jan 2014 | A1 |
20140055481 | Gao | Feb 2014 | A1 |
20140063039 | Drzaic | Mar 2014 | A1 |
20140071102 | Pieper et al. | Mar 2014 | A1 |
20140152703 | Yang et al. | Jun 2014 | A1 |
20140285477 | Cho | Sep 2014 | A1 |
20140307007 | Cho et al. | Oct 2014 | A1 |
20140368483 | Zhang et al. | Dec 2014 | A1 |
20150070337 | Bell | Mar 2015 | A1 |
20150070402 | Shah et al. | Mar 2015 | A1 |
20150348502 | Marcu et al. | Dec 2015 | A1 |
20160140889 | Wu et al. | May 2016 | A1 |
20160180780 | Chen et al. | Jun 2016 | A1 |
20170011678 | Cao et al. | Jan 2017 | A1 |
Number | Date | Country |
---|---|---|
101765878 | Jun 2010 | CN |
201796510 | Apr 2011 | CN |
201893105 | Jul 2011 | CN |
103634581 | Mar 2014 | CN |
103957345 | Jul 2014 | CN |
2172925 | Apr 2010 | EP |
2299723 | Mar 2011 | EP |
2557557 | Feb 2013 | EP |
2341033 | Mar 2000 | GB |
09186896 | Jul 1997 | JP |
H09098301 | Aug 1997 | JP |
11212056 | Aug 1999 | JP |
2002041017 | Feb 2002 | JP |
2002262119 | Sep 2002 | JP |
2002290979 | Oct 2002 | JP |
2002320233 | Oct 2002 | JP |
2002325260 | Nov 2002 | JP |
2003150099 | May 2003 | JP |
2004531716 | Oct 2004 | JP |
2008205542 | Sep 2008 | JP |
2008206067 | Sep 2008 | JP |
2009244340 | Oct 2009 | JP |
2009267967 | Nov 2009 | JP |
2010-113301 | May 2010 | JP |
2010217133 | Sep 2010 | JP |
2010128218 | Oct 2010 | JP |
2299723 | Mar 2011 | JP |
2011205275 | Oct 2011 | JP |
2012027104 | Feb 2012 | JP |
1020070014813 | Feb 2007 | KR |
20120119717 | Oct 2012 | KR |
1020140094561 | Jul 2014 | KR |
20140116607 | Oct 2014 | KR |
200826038 | Jun 2008 | TW |
2013058945 | Apr 2013 | WO |
2014068830 | May 2014 | WO |
2015038407 | Mar 2015 | WO |
Number | Date | Country | |
---|---|---|---|
20190139512 A1 | May 2019 | US |
Number | Date | Country | |
---|---|---|---|
62096188 | Dec 2014 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 15388416 | Dec 2016 | US |
Child | 16194084 | US | |
Parent | 14673667 | Mar 2015 | US |
Child | 15388416 | US |