Patent Grant
10884489
This application is the U.S. national phase of PCT Application No. PCT/CN2017/096774 filed on Aug. 10, 2017, which claims priority to Chinese Patent Application No. 201710001943.X filed on Jan. 3, 2017, which are incorporated herein by reference in their entireties.
The present disclosure relates to the field of display technology, in particular to a processing circuit of a display panel, a display method, and a display device.
Along with the continuous development of the display technology, the definition of an image is highly demanded, so there has currently been a trend to provide a high resolution display device.
Usually, resolution refers to the number of pixels of the display device, e.g., in the case that the display device includes 1920*1080 pixels, its resolution is just 1920*1080. Points, lines and planes of the image each consists of the pixels, so the more the pixels, the finer the image, and the more the information displayed by the display device. Hence, a display effect of the image directly depends on a size of the resolution, which is one of the important indexes for the performance of the display device. However, in the case that the high resolution is provided, a large quantity of processing resources may inevitably be occupied and a data transmission volume may increase, and thereby the power consumption of the display device may increase significantly.
In one aspect, the present disclosure provides in some embodiments a processing circuit of a display panel, including: a line-of-sight acquisition module configured to track an eyeball of each eye, to determine a concern region of line of sight on the display panel and a region other than the concern region; a control module configured to acquire original image data of an image to be displayed on the display panel, subject first original image data corresponding to the concern region and/or second original image data corresponding to the other region to treatment, and output first image generation data corresponding to the concern region and second image generation data corresponding to the other region and having a resolution smaller than the first image generation data; and a display signal output module configured to output a display signal to the display panel in accordance with the first image generation data and the second image generation data.
In a possible embodiment of the present disclosure, the line-of-sight acquisition module includes: a face identification unit configured to extract a face through a face detection algorithm, acquire a face region, and determine a deflection angle of the face; an eye identification unit configured to detect an eye feature at the face region, to acquire an eye region and determine a distance between the eye and the display panel; an offset calculation unit configured to determine a center of a pupil of each eye at the eye region, and calculate a movement amount of the center of the pupil of each eye; a direction vector calculation unit configured to calculate a direction vector of the line of sight of each eye in accordance with the movement amount of the center of the pupil of each eye and the deflection angle of the face; and a concern region determination unit configured to calculate the concern region of the line of sight and the region other than the concern region in accordance with the direction vector of the line of sight and space coordinates of each eye relative to the display panel.
In a possible embodiment of the present disclosure, the display panel includes subpixels arranged in a plurality of rows, the subpixels in two adjacent rows are arranged in such a staggered manner that a staggered distance between each two adjacent rows is equal to a half width of the subpixel, and each subpixel in one row is in a color different from two subpixels in the adjacent row closest to the subpixel. The control module includes: an original image data acquisition unit configured to acquire the original image data of the image to be displayed on the display panel, and determine the first original image data corresponding to the concern region and the second original image data corresponding to the other region; and a processing unit configured to, in the case that a resolution of the original image data is greater than a first threshold, subject the first original image data to BV3 algorithm treatment to acquire the first image generation data corresponding to the concern region and subject an image corresponding to the second original image data to reduction treatment to acquire the second image generation data corresponding to the other region, and in the case that the resolution of the original image data is smaller than a second threshold smaller than the first threshold, enlarge an image corresponding to the first original image data, subject the enlarged image to the BV3 algorithm treatment to acquire the first image generation data corresponding to the concern region, and directly output the second original image data as the second image generation data wherein the BV3 algorithm is an algorithm configured to performed weighted calculation on image data of original adjacent pixels to acquire image generation data having a resolution half of the original image data.
In a possible embodiment of the present disclosure, after the processing unit has subjected the image corresponding to the second original image data to the reduction treatment to acquire the second image generation data corresponding to the other region, the display signal output module is further configured to output to the display panel a display signal for displaying the second image generation data in a resolution-reduced, SmartView mode.
In another aspect, the present disclosure, the present disclosure provides in some embodiments a display device including a display panel and the above-mentioned processing circuit.
In a possible embodiment of the present disclosure, the control module is integrated into a Central Processing Unit (CPU) of the display device and the display signal output module is integrated into a driver Integrated Circuit (IC) of the display panel, or both the control module and the display signal output module are integrated into the driver IC of the display panel.
In a possible embodiment of the present disclosure, a base substrate of the display panel is a silicon substrate, and the driver IC of the display panel is integrated into the silicon substrate.
In a possible embodiment of the present disclosure, the display device is a head-mounted display device.
In yet another aspect, the present disclosure provides in some embodiments a display method for use in the above-mentioned display panel, including steps of: tracking an eyeball of each eye to determine a concern region of line of sight on the display panel and a region other than the concern region; acquiring original image data of an image to be displayed on the display panel, subjecting first original image data corresponding to the concern region and/or second original image data corresponding to the other region to treatment, and outputting first image generation data corresponding to the concern region and second image generation data corresponding to the other region and having a resolution smaller than the first image generation data; and outputting a display signal to the display panel in accordance with the first image generation data and the second image generation data.
In a possible embodiment of the present disclosure, the step of tracking the eyeball of each eye to determine the concern region of the line of sight and the region other than the concern region includes: extracting a face through a face detection algorithm, acquiring a face region, and determining a deflection angle of the face; detecting an eye feature at the face region, to acquire an eye region and determine a distance between the eye and the display panel; determining a center of a pupil of each eye at the eye region, and calculating a movement amount of the center of the pupil of each eye; calculating a direction vector of the line of sight of each eye in accordance with the movement amount of the center of the pupil of each eye and the deflection angle of the face; and calculating the concern region of the line of sight and the region other than the concern region in accordance with the direction vector of the line of sight and space coordinates of each eye relative to the display panel.
In a possible embodiment of the present disclosure, the display panel includes subpixels arranged in a plurality of rows, the subpixels in two adjacent rows are arranged in such a staggered manner that a staggered distance between each two adjacent rows is equal to a half width of the subpixel, and each subpixel in one row is in a color different from two subpixels in the adjacent row closest to the subpixel. The step of acquiring original image data of an image to be displayed on the display panel, subjecting first original image data corresponding to the concern region and/or second original image data corresponding to the other region to treatment, and outputting first image generation data corresponding to the concern region and second image generation data corresponding to the other region includes: acquiring the original image data of the image to be displayed on the display panel, and determining the first original image data corresponding to the concern region and the second original image data corresponding to the other region; in the case that a resolution of the original image data is greater than a first threshold, subjecting the first original image data to BV3 algorithm treatment to acquire the first image generation data corresponding to the concern region and subjecting an image corresponding to the second original image data to reduction treatment to acquire the second image generation data corresponding to the other region; and in the case that the resolution of the original image data is smaller than a second threshold smaller than the first threshold, enlarging an image corresponding to the first original image data, subjecting the enlarged image to the BV3 algorithm treatment to acquire the first image generation data corresponding to the concern region, and directly outputting the second original image data as the second image generation data.
In a possible embodiment of the present disclosure, subsequent to the step of subjecting the image corresponding to the second original image data to the reduction treatment to acquire the second image generation data corresponding to the other region, the step of outputting the display signal to the display panel in accordance with the first image generation data and the second image generation data includes displaying the second image generation data in a resolution-reduced, SmartView mode.
In a possible embodiment of the present disclosure, the step of displaying the second image generation data in the resolution-reduced, SmartView mode includes applying a voltage to gate lines connected to subpixels in two adjacent rows, subjecting every two adjacent subpixels in an identical color in the subpixels in the two adjacent rows to identical treatment, and enabling two adjacent subpixels in a vertical direction to emit light at an identical brightness value.
According to the embodiments of the present disclosure, through determining the concern region of the line of sight on the display panel, it is able to adjust the resolutions of the images at different regions, i.e., to enable the resolution of the image corresponding to the concern region to be greater than that of the image corresponding to the other region. As a result, it is able to reduce the data processing burden and the data transmission volume of the display device while ensuring the display quality and the user experience, thereby to reduce the power consumption of the display device and extend a battery life of the display device.
In order to make the objects, the technical solutions and the advantages of the present disclosure more apparent, the present disclosure will be described hereinafter in a clear and complete manner in conjunction with the drawings and embodiments. Obviously, the following embodiments merely relate to a part of, rather than all of, the embodiments of the present disclosure, and based on these embodiments, a person skilled in the art may, without any creative effort, obtain the other embodiments, which also fall within the scope of the present disclosure.
Unless otherwise defined, any technical or scientific term used herein shall have the common meaning understood by a person of ordinary skills. Such words as “first” and “second” used in the specification and claims are merely used to differentiate different components rather than to represent any order, number or importance. Similarly, such words as “one” or “one of” are merely used to represent the existence of at least one member, rather than to limit the number thereof. Such words as “connect” or “connected to” may include electrical connection, direct or indirect, rather than to be limited to physical or mechanical connection. Such words as “on”, “under”, “left” and “right” are merely used to represent relative position relationship, and when an absolute position of the object is changed, the relative position relationship will be changed too.
The present disclosure provides in some embodiments a processing circuit of a display panel, a display method and a display device, to reduce the power consumption of the display device while ensuring the display quality, thereby to make a compromise between the display quality and power consumption.
The present disclosure provides in some embodiments a processing circuit of a display panel which, as shown in
Here, the line-of-sight acquisition module may be implemented by such hardware as an image acquisition device (e.g., a camera) and a relevant processing circuit (e.g., chip or CPU). The control module may be implemented by such hardware as a processing circuit (e.g., chip or CPU). The display signal output module may be implemented by a display device (e.g., a Liquid Crystal Display (LCD) or an Organic Light-Emitting Diode (OLED) display device).
According to the embodiments of the present disclosure, through determining the concern region of the line of sight on the display panel, it is able to adjust the resolutions of the images at different regions, i.e., to enable the resolution of the image corresponding to the concern region to be greater than that of the image corresponding to the other region. As a result, it is able to reduce the data processing burden and the data transmission volume of the display device while ensuring the display quality and the user experience, thereby to reduce the power consumption of the display device and extend a battery life of the display device.
In a possible embodiment of the present disclosure, as shown in
The face identification unit 111 is configured to extract a face through a face detection algorithm, acquire a face region, and determine a deflection angle of the face. To be specific, information about a face image of a user may be acquired by an image sensor, and then the information about the face image may be processed by a processor, to acquire the face region in the image. Here, the methods for acquiring the face image and acquiring the face region from the face image are known in the art, and thus will not be particularly defined herein.
The eye identification unit 112 is configured to detect an eye feature at the face region, to acquire an eye region and determine a distance between the eye and the display panel. To be specific, upon the determination of the eye region, information about pupils of the eyes may be extracted, and then profile analysis may be performed on the information, to determine a center of the pupil of each eye. In addition, upon the determination of the eye region, space coordinates of each eye relative to the display panel may be calculated, and then the distance between each eye and the display panel may be calculated in accordance with the space coordinates of each eye relative to the display panel. Here, the detection of the eye feature at the face region, the acquisition of the eye region, the calculation of the space coordinates of each eye relative to the display panel, and the calculate of the distance between each eye and the display panel may be performed through a processor using known methods, which will not be particularly defined herein.
The offset calculation unit 113 is configured to determine the center of the pupil of each eye at the eye region, and calculate a movement amount of the center of the pupil of each eye.
The direction vector calculation unit 114 is configured to calculate a direction vector of the line of sight of each eye in accordance with the movement amount of the center of the pupil of each eye and the deflection angle of the face.
The concern region determination unit 115 is configured to calculate the concern region of the line of sight and the region other than the concern region in accordance with the direction vector of the line of sight and the space coordinates of each eye relative to the display panel.
The above description has been given so as to facilitate the understanding of an eye tracking mode, and it should be appreciated that, any other eye tracking mode may also be applicable, which will not be particularly defined herein. Here, the offset calculation unit 113, the direction vector calculation unit 114 and the concern region determination unit 115 may be implemented by such hardware as a processing circuit (e.g., chip or CPU), and they are merely used to perform simple spatial geometric operation, which will not be particularly defined herein.
In a possible embodiment of the present disclosure, as shown in
A BV3 algorithm is used to treat an image, whose original resolution is double the number of actual subpixels in a horizontal direction, in such a manner that the treated image is capable of being displayed on the display panel, to acquire an effect similar to the original high resolution based on the principle that the human eye is more sensitive to brightness, thereby to improve the resolution of the display image.
The BV3 algorithm will be described hereinafter.
In the case that a sequence of subpixels corresponding to an image having the original resolution is shown in the following table
and a sequence of actual subpixels corresponding to an image to be displayed is shown in the following table
corresponding conversion equations will be described as follows.
For the subpixels in odd-numbered rows, R1=∂r1+∂2r2, R2=∂r3+∂r4, . . . , where ∂1+∂2=1; G1=β1g1+β2g2, G2=β1g3+β2g4, . . . , where β1+β2=1; and B1=r1b1+r2b2, B2=r1b3+r2b4, . . . , where r1+r2=1. Coefficients ∂1, ∂2, β1, β2, r1 and r2 may be determined in accordance with the practical need, or in accordance with a debugging result. For example, each of them may be 0.5.
For the subpixels in even-numbered rows and at boundaries, R4=r8, and the red (R) subpixels at a right boundary may be treated in a similar way; G4=g8, and the green (G) subpixels at the right boundary may be treated in a similar way; and B3=b5, and the blue (B) subpixels at a left boundary may be treated in a similar way.
For the subpixels in the even-numbered rows and not at boundaries, R3=∂3r6+∂4r7, . . . , where ∂3+∂4=1; G3=β3g6+β4g7, . . . , where β3+β4=1; and B4=r3b7+r4b6, . . . , where r3+r4=1. The coefficients ∂1, ∂2, β1, β2, r1 and r2 may be determined in accordance with the practical need, or in accordance with a debugging result. For example, each of them may be 0.5.
As mentioned above, the conversion of the actual pixels are performed using the BV3 algorithm. In other words, the BV3 algorithm is used to performed weighted calculation on the image data of the original adjacent pixels, to acquire the image generation data having a resolution half of the original image data.
In a possible embodiment of the present disclosure, after the processing unit has subjected the image corresponding to the second original image data to the reduction treatment to acquire the second image generation data corresponding to the other region, the display signal output module is further configured to output to the display panel a display signal for displaying the second image generation data in a resolution-reduced, SmartView mode. As shown in
The present disclosure further provides in some embodiments a display device including a display panel and the above-mentioned processing circuit. The display device may be any product or member having a display function, such as a television, a display, a digital photo frame, a mobile phone or a flat-panel computer. The display device may further include a flexible circuit board, a printed circuit board and a back plate. The display panel may be an LCD panel or an OLED panel.
In a possible embodiment of the present disclosure, the control module is integrated into a CPU of the display device and the display signal output module is integrated into a driver IC of the display panel, or both the control module and the display signal output module are integrated into the driver IC of the display panel.
In a possible embodiment of the present disclosure, upon the acquisition of the line of sight, an image sensor may transmit information about the line of sight to the CPU of the display device. Next, the CPU of the display device may acquire the original image data of the image to be displayed on the display panel, subject the acquired original image data in accordance with the information about the line of sight to acquire the first image generation data and the second image generation data, and output the first image generation data and the second image generation data to the driver CI of the display panel, so that the driver IC of the display panel may display the image in accordance with the first image generation data and the second image generation data. The information about the line of sight may include information about the concern region of the line of sight and the region other than the concern region on the display panel.
In another possible embodiment of the present disclosure, upon the acquisition of the line of sight, the image sensor may transmit the information about the line of sight to the CPU of the display device. Next, the CPU of the display device may transmit the information about the line of sight to the driver IC of the display panel. Then, the driver IC of the display panel may acquire the original image data of the image to be displayed on the display panel, subject the acquired original image data to the treatment in accordance with the information about the line of sight to acquire the first image generation data and the second image generation data, and display the image in accordance with the first image generation data and the second image generation data. The information about the line of sight may include information about the concern region of the line of sight and the region other than the concern region on the display panel.
In a possible embodiment of the present disclosure, a base substrate of the display panel is a silicon substrate, and the driver IC of the display panel is integrated into the silicon substrate. Due to a very strong information storage capability of the silicon substrate, the driver IC of the display panel may be integrated into the silicon substrate, to simplify the structure of the display device.
In a possible embodiment of the present disclosure, the display device is a head-mounted display device. In the case that the head-mounted display device is used, through determining the concern region of the line of sight on the display panel, it is able to adjust the resolutions of the images at different regions, i.e., to output a high-resolution image at the concern region and output a low-resolution image at the other region. As a result, it is able to provide the images with different resolutions while meeting the requirements of the user, reduce the power consumption of the driver IC of the display pane, and reduce the data processing burden and the data transmission volume of the head-mounted display device, thereby to reduce the power consumption of the head-mounted display device and extend a battery life of the head-mounted display device.
The present disclosure further provides in some embodiments a display method for use in the above-mentioned display panel. As shown in
According to the embodiments of the present disclosure, through determining the concern region of the line of sight on the display panel, it is able to adjust the resolutions of the images at different regions, i.e., to enable the resolution of the image corresponding to the concern region to be greater than that of the image corresponding to the other region. As a result, it is able to reduce the data processing burden and the data transmission volume of the display device while ensuring the display quality and the user experience, thereby to reduce the power consumption of the display device and extend a battery life of the display device.
In a possible embodiment of the present disclosure, as shown in
Step 601: extracting a face through a face detection algorithm, acquiring a face region, and determining a deflection angle of the face. Here, information about a face image of a user may be acquired by an image sensor
Step 602: detecting an eye feature at the face region, to acquire an eye region and determine a distance between the eye and the display panel. To be specific, upon the determination of the eye region, information about pupils of the eyes may be extracted, and then profile analysis may be performed on the information, to determine a center of the pupil of each eye. In addition, upon the determination of the eye region, space coordinates of each eye relative to the display panel may be calculated, and then the distance between each eye and the display panel may be calculated in accordance with the space coordinates of each eye relative to the display panel.
Step 603: determining the center of the pupil of each eye at the eye region, and calculating a movement amount of the center of the pupil of each eye.
Step 604: calculating a direction vector of the line of sight of each eye in accordance with the movement amount of the center of the pupil of each eye and the deflection angle of the face.
Step 605: calculating the concern region of the line of sight and the region other than the concern region in accordance with the direction vector of the line of sight and the space coordinates of each eye relative to the display panel.
The above description has been given to facilitate the understanding of an eye tracking mode, and it should be appreciated that, any other eye tracking mode may also be applicable, which will not be particularly defined herein.
In a possible embodiment of the present disclosure, as shown in
Step 701: acquiring the original image data of the image to be displayed on the display panel, and determining the first original image data corresponding to the concern region and the second original image data corresponding to the other region.
Step 702: determining whether or not a resolution of the original image data is greater than a first threshold or smaller than a second threshold which is smaller than the firs threshold, in the case that the resolution of the original image data is greater than the first threshold, proceeding to Step 703, and in the case that the resolution of the original image data is smaller than the second threshold, proceeding to Step 704.
Step 703: subjecting the first original image data to BV3 algorithm treatment to acquire the first image generation data corresponding to the concern region and subjecting an image corresponding to the second original image data to reduction treatment to acquire the second image generation data corresponding to the other region.
Step 704: enlarging an image corresponding to the first original image data, subjecting the enlarged image to the BV3 algorithm treatment to acquire the first image generation data corresponding to the concern region, and directly outputting the second original image data as the second image generation data.
As shown in
In a possible embodiment of the present disclosure, subsequent to the step of subjecting the image corresponding to the second original image data to the reduction treatment to acquire the second image generation data corresponding to the other region, the step of outputting the display signal to the display panel in accordance with the first image generation data and the second image generation data includes displaying the second image generation data in a resolution-reduced, SmartView mode. In
The functional members described in the specification are referred to as modules or units, to emphasize the independence of the implementation in a more particular manner.
According to the embodiments of the present disclosure, the modules/units may be implemented by software or circuits, so as to be executed by various processors. For example, an identified, executable code module may include one or more physical or logical blocks including computer instructions, and the module may be constructed as an image, a process or a function. Even so, the executable codes of the identified modules are unnecessary to be physically located together, but may include different instructions stored in different locations. In the case that these instructions are logically combined together, they form the modules and achieve the prescribed purposes of the modules.
Actually, the executable code module may be a single instruction or a plurality of instructions, and may even be distributed at different code segments, in different programs, or across a plurality of memory devices. Also, operational data may be identified in the modules, implemented in any appropriate form, and organized in any data structure of an appropriate type. The operational data may be collected as a single data set, or distributed at different locations (including different memory devices), and may be at least partially present in a system or network merely as an electronic signal.
In the case that the modules are implemented by software, considering the current hardware level, a person skilled in the art may build a corresponding hardware circuit to achieve the corresponding function in the case of taking no account of the cost. The hardware circuit includes a conventional very-large-scale integration (VLSI) circuit, a gate array, an existing semiconductor such as a logic chip and a transistor, or other discrete components. The modules may further be implemented by a programmable hardware device, such as a field-programmable gate array, a programmable array logic device and a programmable logic device.
In the embodiments of the present disclosure, the order of the steps is not limited to the serial numbers thereof. For a person skilled in the art, any change in the order of the steps shall also fall within the scope of the present disclosure if without any creative effort.
The above are merely the preferred embodiments of the present disclosure, but the present disclosure is not limited thereto. Obviously, a person skilled in the art may make further modifications and improvements without departing from the spirit of the present disclosure, and these modifications and improvements shall also fall within the scope of the present disclosure.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2017 1 0001943 | Jan 2017 | CN | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2017/096774 | 8/10/2017 | WO | 00 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2018/126686 | 7/12/2018 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 4513317 | Ruoff, Jr. | Apr 1985 | A |
| 10387782 | Kon | Aug 2019 | B2 |
| 20020141614 | Lin | Oct 2002 | A1 |
| 20050259035 | Iwaki | Nov 2005 | A1 |
| 20120069243 | Beemster | Mar 2012 | A1 |
| 20120154557 | Perez | Jun 2012 | A1 |
| 20130106674 | Wheeler | May 2013 | A1 |
| 20130335404 | Westerinen | Dec 2013 | A1 |
| 20140125558 | Miyajima | May 2014 | A1 |
| 20190361658 | Shi | Nov 2019 | A1 |
| Number | Date | Country |
|---|---|---|
| 102547123 | Jul 2012 | CN |
| 103207664 | Jul 2013 | CN |
| 103902978 | Jul 2014 | CN |
| 103974115 | Aug 2014 | CN |
| 106531073 | Mar 2017 | CN |
| Entry |
|---|
| International Search Report and Written Opinion for Application No. PCT/CN2017/096774, dated Oct. 26, 2017, 10 Pages. |
| First Office Action for Chinese Application No. 201710001943.X, dated Apr. 8, 2018, 7 Pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20200218340 A1 | Jul 2020 | US |
