Patent Grant
10424236
This patent application claims priority of Chinese Patent Application No. 201610345912.1, filed on May 23, 2016, the entire content of which is incorporated by reference herein.
The text of the present disclosure relates to the field of image display, and in particular, to a method, apparatus and system for displaying an image.
Flat display devices are widely used in people's daily lives, and in recent years, with the continuous development of science and technology, curved surface display devices having a curved surface display panel are gradually appearing on the market.
Compared to the flat display device, since the curved surface display panel of the curved surface display device more matches with retina having a curved surface, a curved surface image rendered on the curved surface display panel may be more intuitively mapped to the viewer's brain, without the need to switch the plane image, and this advantage can provide users with more natural, comfortable visual experiences, along with a better sense of immersion.
However, the current existing curved surface display panel is mainly implemented by physically bending OLED or physically bending LCD and backlight, and since the process and technology for physically bending OLED or LCD or backlight to produce the curved surface display panel are not yet mature, compared to the traditional flat display panel, its production costs are higher. This limits the promotion and application of the curved surface display device.
The content of the present disclosure is provided below to provide a basic understanding of some aspects of the present disclosure. The content of the present disclosure is not a broad overview of the present disclosure. It is neither used to identify the main or critical part of the present disclosure nor to define the scope of the present disclosure, but merely provides some concepts of the present disclosure in a simplified form as a prelude to the subsequent embodiments.
Embodiments of the present disclosure provide a display method capable of displaying an image having a curved surface effect on a flat display panel.
According to an aspect of the present disclosure, there is provided a method for displaying an image, comprising: acquiring a position of a visual center point of a viewer relative to a flat display panel; generating a curved surface based on the position of the visual center point relative to the flat display panel, wherein a line between the visual center point and any point on the curved surface passes through a plane formed by the flat display panel, and intersects the curved surface once; generating a pixel grid on the curved surface based on resolution of the flat display panel; determining color information and luminance information of a pixel in the pixel grid when the image is displayed on the pixel grid; determining a valid pixel set in the flat display panel corresponding to the pixel grid, wherein a point of intersection of an extension line of the line between the visual center point and any point in the plane formed by the valid pixel set with the curved surface is in the pixel grid; rendering the valid pixel set of the flat display panel based on the color information and luminance information of the pixel grid. The method for displaying an image converts a flat display panel image into a curved surface display effect image corresponding to the curved surface so that a display image of the flat display panel has a curved surface display effect corresponding to the curved surface.
According to another aspect of the present disclosure, there is provided an apparatus for displaying an image, comprising: an acquisition module configured to acquire a position of a visual center point of a viewer relative to a flat display panel; a curved surface generation module configured to generate a curved surface based on the position of the visual center point relative to the flat display panel, wherein a line between the visual center point and any point on the curved surface passes through a plane formed by the flat display panel, and intersects the curved surface once; a pixel grid generation module configured to generate a pixel grid on the curved surface based on resolution of the flat display panel; a grid color and luminance determination module configured to determine the color information and luminance information of the pixels in the pixel grid when the image is displayed on the pixel grid; a valid pixel determination module configured to determine a valid pixel set corresponding to the pixel grid in the flat display panel, wherein a point of intersection of an extension line of the line between the visual center point and any point in the plane formed by the valid pixel set is in the pixel grid; a valid pixel rendering module configured to render the valid pixel set of the flat display panel based on the color information and the luminance information of the pixel grid. The apparatus for displaying an image converts a flat display panel image into a curved surface display effect image corresponding to the curved surface so that a display image of the flat display panel has a curved surface display effect corresponding to the curved surface.
According to still another aspect of the present disclosure, there is provided an apparatus for displaying an image, comprising: at least one processor, and at least one memory coupled to the at least one processor and storing a program capable of being executed by the at least one processor, wherein the at least one processor executes the program for: acquiring a position of a visual center point of a viewer relative to a flat display panel; generating a curved surface based on the position of the visual center point relative to the flat display panel, wherein a line between the visual center point and any point on the curved surface passes through a plane formed by the flat display panel and intersects the curved surface once; generating a pixel grid on the curved surface based on resolution of the flat display panel; determining color information and luminance information of a pixel in the pixel grid when the image is displayed on the pixel grid; determining a valid pixel set in the flat display panel corresponding to the pixel grid, wherein a point of intersection of an extension line of the line between the visual center point and any point in the plane formed by the valid pixel set with the curved surface is in the pixel grid; rendering the valid pixel set of the flat display panel based on the color information and the luminance information of the pixel grid. The apparatus for displaying an image converts a flat display panel image into a curved surface display effect image corresponding to the curved surface so that a display image of the flat display panel has a curved surface display effect corresponding to the curved surface.
According to yet another aspect of the present disclosure, there is provided a system for displaying an image, comprising: a flat display panel; and an apparatus for displaying an image according to any one of the preceding, which is coupled to the flat display panel, as described above.
Objects, features and advantages of the present disclosure will become apparent from the following detailed description of embodiments of the present disclosure when taken in conjunction with the accompanying drawings, in which:
Embodiments of the present disclosure will now be described more comprehensively with reference to the accompanying drawings, in which some, but not all, of the embodiments of the present disclosure are shown. Of course, the present disclosure may be embodied in many different forms and should not be construed as limiting the embodiments set forth herein; rather, these embodiments are provided so that the present disclosure will satisfy applicable legal requirements. Wherein, similar reference numbers refer to similar elements.
It should also be noted that in some alternative embodiments, the proposed functions/behaviors may occur in a different order than that shown in the figures. For example, two graphs shown continuously may actually be executed substantially simultaneously, or may sometimes be executed in reverse order, depending on the function/behavior involved.
Unless expressly stated otherwise in the context, the singular form of a word used in the description and the appended claims includes the plural form, and vice versa. Thus, when the singular is referred to, the plural of the corresponding term is usually included. Similarly, the words “containing” and “comprising” will be construed as being included and not exclusively. Likewise, the terms “comprising” and “or” should be construed as being included, unless expressly prohibited herein. Where the term “instance” is used herein, particularly when it is located after a set of terms, the “instance” is merely exemplary and descriptive and should not be considered exclusive or comprehensive.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those understood by those skilled in the art to which the exemplary embodiments pertain. It is also to be understood that terms (such as those defined in the general dictionary) should be interpreted as having meaning consistent with their meaning in the context of the relevant field and are not interpreted in an idealized or excessively formalized way, unless expressly defined herein.
The descriptions of part of the present disclosure and the corresponding details are provided in terms of symbolic representations and software or algorithms for operation of data bits in a computer memory. These descriptions and representations are the means by which a person skilled in the art will readily convey the content of their work to other technical staff in the art. As used herein, and as commonly used, an algorithm may be considered as a self-consistent sequence of steps leading to a desired result. The steps are those steps that require physical operations of physical quantities. Usually, although not necessary, these quantities are in the form of optical, electrical, or magnetic signals that can be stored, transmitted, combined, compared and manipulated. Often proven to be convenient, these signals, for general usage of the principle, are called bits, values, elements, symbols, words, items, numbers and so on.
In the following description, exemplary embodiments may be described with reference to behaviors and symbolic representations (e.g., in the form of a flowchart) of operations that may be implemented as program modules or functional processes including routines, programs, objects, components, data results, etc., which perform particular tasks or implement particular abstract data types, and may be implemented at an existing network unit or control node using existing hardware. Such existing hardware may include one or more central processing units (CPUs), digital signal processors (DSPs), application specific integrated circuits, field programmable gate arrays (FPGAs), and the like.
However, it should be remembered that all these and similar terms are related to appropriate physical quantities and are only convenient labels applied to these quantities. The term, unless particularly stated, or apparent from a discussion, such as “processing” or “calculating” or “determining” or “displaying”, refers to the behavior and processing of a computer system, or a similar electronic computing device, which operates data in the register and memory of the computer system represented as physical and electronic quantities and converts them into other data similarly represented as physical quantities in the computer system memory or registers or other such information storage devices, transmission or display devices.
It should also be noted that aspects of the software implementation of the present disclosure are typically encoded on a certain program storage medium or implemented on some types of transmission media. The program storage medium may be magnetic (e.g., a floppy disk or hard disk drive) or optical (e.g., compact disk read only memory or “CD ROM”), and may be read-only or random access. Similarly, the transmission media may be a twisted pair, a coaxial cable, an optical fiber, or some other suitable transmission medium known in the art. The present disclosure is not limited by these aspects of any given scheme.
A known method or apparatus for displaying an image uses a display panel which exhibits a curved shape. The application of the curved-surface display is limited due to the high manufacturing cost of a curved surface display panel, especially compared to a conventional flat display panel.
Embodiments of the present disclosure are designed to address this problem, and in particular, embodiments of the present disclosure may be used, but not limited to, to display an image having a curved surface display effect.
In an embodiment of the present disclosure, the position of the visual center point relative to the flat display panel may be manually entered by the user, such as through an input device coupled to the flat panel display, such as a button group or a keyboard coupled to the flat panel display.
In an embodiment of the present disclosure, the position of the visual center point relative to the flat display panel may be acquired from a detector that tracks the eye movement of the viewer, and the detector may be one or more cameras, the acquisition may be for example to acquire the eye position of the viewer from the images captured by a plurality of cameras located at different positions and determine the position of the visual center point of the viewer relative to the flat display panel based on the known relative positional relationships among the plurality of cameras.
In step 220, a curved surface is generated based on the position of the visual center point relative to the flat display panel, wherein a line between the visual center point and any point on the curved surface passes through a plane formed by the flat display panel, and intersects said curved surface once. Referring to the example of
In an embodiment of the present disclosure, the curved surface is a part of a side surface of a cylinder. In an embodiment of the present disclosure, a height of the cylinder is the same as a length of a longitudinal side of the flat display panel, and a direction of a central axis of the cylinder is the same as that of the longitudinal side. In an embodiment of the present disclosure, the side surface of the cylinder passes through two longitudinal sides of the flat display panel. In an embodiment of the present disclosure, a radius of the cylinder may be manually entered by a user, such as through an input device coupled to the flat panel display, such as a button group or a keyboard coupled to the flat panel display. In an embodiment of the present disclosure, the radius of the cylinder may be determined based on the position of the visual center point relative to the flat display panel, for example, the radius of the cylinder may be a distance from the visual center point to the center point of the flat display panel plus the predetermined length, for example 2 cm.
Then in step 230, a pixel grid is generated on the curved surface based on resolution of the flat display panel. In an embodiment of the present disclosure, the resolution of the pixel grid is the same as the resolution of the flat display panel. In an embodiment of the present disclosure, the resolution of the pixel grid is greater than the resolution of the flat display panel, for example, the resolution of the pixel grid is a multiple of the resolution of the flat panel display panel, such as the resolution of the pixel grid is 2 times, 4 times or 8 times the resolution of the flat panel display.
Subsequently, in step 240, color information and luminance information of pixels in the pixel grid are determined when the image is displayed on the pixel grid. The color information and the luminance information of the pixels in the pixel grid may be determined based on any method known to those skilled in the art, for example, the image may be stretched to conform to the resolution of the pixel grid, and the color information and the luminance information of the pixels in the pixel grid are further determined, said method including, but not limited to, operations of the image, such as stretching, scaling, rotating, and the like.
Subsequently, in step 250, a valid pixel set in the flat display panel corresponding to the pixel grid is determined, wherein a point of intersection of an extension line of the line between the visual center point and any point in the plane formed by the valid pixel set with the curved surface is the pixel grid. Referring to
In step 260 of
In step 510, one or more pixels in the pixel grid that overlaps a projection of the valid pixel on the curved surface with respect to the visual center point are determined. Here, one or more pixels in the pixel grid that overlaps the projection of the valid pixel on the curved surface may be determined based on the perspective projection algorithm. Specifically, with reference to
Subsequently, in step 520, an overlapping area of the projection with each of the one or more pixels and a projection distance from the valid pixel to each of the one or more pixels are determined. In an embodiment of the present disclosure, the projection distance may be a distance from the center point of each of the one or more pixels to the center point of the valid pixel.
Subsequently, in step 530, first color information and first luminance information of the pixel in the one or more pixels that has the largest overlapping area with the projection are acquired.
Subsequently, in step 540, second luminance information is determined based on the first luminance information and the projection distance. The determination of the second luminance information may be based on the first luminance information and a attenuation luminance of the backlight source of the flat display panel at the projection distance. In an embodiment of the present disclosure, the attenuation luminance at the projection distance is a luminance that is attenuated during the light emitted by the backlight source propagates the projection distance in the air. In an embodiment of the present disclosure, the second luminance is the first luminance subtracting the attenuation luminance at the projection distance.
Finally, in step 550, the valid pixel is rendered using the first color information and the second luminance information.
Steps 610-640 are the same as steps 510-540 in
In step 650, color information and luminance information of a neighboring valid pixel adjacent to the valid pixel are acquired. Here, the neighboring valid pixels are, for example, eight valid pixels adjacent to the valid pixel, or four valid pixels, upper, lower, left and right, adjacent to the valid pixel.
Then in step 660, the first color information and the second luminance information are adjusted according to the color information and the luminance information for the neighboring valid pixel. The adjustment herein may be, for example, an interpolation operation based on the color information and the luminance information of the neighboring valid pixels, and the calculated color information and luminance information may serve as the adjusted first color information and second luminance information of the valid pixel, or the color information and luminance information of the pixel closest to the average color value of the neighboring valid pixels may serve as the adjusted first color information and second luminance information of the valid pixel.
In step 670, the valid pixel is rendered by using the adjusted first color information and second luminance information.
Preferably, in the method according to the embodiment of the present disclosure, pixels other than the valid pixel set in the flat display panel may be rendered to coincide with color or hue of a bezel of a planar image display device, or in the case that the flat display panel is a transparent display panel, the other pixels are maintained in a transparent state. Here, the transparent display panel refers to a display panel through which the user may view objects located therebehind when the pixels thereon are not rendered.
The apparatus 700 includes an acquisition module 710 configured to acquire a position of the visual center point relative to the flat display panel. In an embodiment of the present disclosure, the visual center point may be pre-stored in the acquisition module 710. In an embodiment of the present disclosure, the acquisition module 710 may receive the visual center point from an input device (e.g., a button group or a keyboard coupled to a flat display panel) or from an eye movement analysis module, which is coupled to a detector for tracking the eye movement of the viewer and configured to analyze a position of the viewer's eye based on an image received from the detector so as to determine the visual center point and to provide the visual center point to the acquisition module 710, wherein the detector is, for example, one or more cameras.
The acquired visual center point is forwarded to a curved surface generation module 720, which is configured to generate a curved surface based on the position of the visual center point relative to the flat display panel, wherein a line between the visual center point and any point on the curved surface passes through a plane formed by the flat display panel, and intersects the curved surface once.
The generated curved surface is forwarded to a pixel grid generation module 730 which is configured to generate a pixel grid on the curved surface based on resolution of the flat display panel.
The generated pixel grid is forwarded to a grid color and luminance determination module 740, which is configured to determine color information and luminance information of the pixels in the pixel grid when the image is displayed on the pixel grid.
The generated pixel grid is also forwarded to a valid pixel determination module 750, which is configured to determine a valid pixel set in the flat display panel that corresponds to the pixel grid, wherein a point of intersection of an extension line of the line between the visual center point and any point in the plane formed by the valid pixel set with the curved surface is in the pixel grid.
Subsequently, a valid pixel rendering module 760 receives the determined valid pixel set, the pixel grid, and the color information and luminance information of the pixels in the pixel grid, and renders the valid pixel set of the flat display panel based on the pixel grid and the color information and the luminance information of the pixel grid.
An overlapping determination sub-module 810 is configured to determine one or more pixels in the pixel grid that overlap a projection of the valid pixel on the curved surface with respect to the visual center point. Here, the one or more pixels in the pixel grid that overlap the projection of the valid pixel on the curved surface may be determined based on a perspective projection algorithm.
After one or more pixels overlapping the projection are determined, an area and distance sub-module 820 determines an overlapping area of the projection with each of the one or more pixels and a projection distance from the valid pixel to each of the one or more pixels. In an embodiment of the present disclosure, the projection distance may be a distance from a center point of each of the one or more pixels to a center point of the valid pixel.
Then, a color and luminance sub-module 830 acquires first color information and first luminance information of the pixel among the one or more pixels that has the largest overlapping area with the projection.
A second luminance sub-module 840 determines second luminance information based on the first luminance information and the projection distance. In an embodiment of the present disclosure, the second luminance information may be determined based on the first luminance information and the attenuation luminance of the backlight source of the flat display panel at the projection distance. In an embodiment of the present disclosure, the attenuation luminance at the projection distance is a luminance that is attenuated during the light emitted by the backlight source propagates the projection distance in the air. In an embodiment of the present disclosure, the second luminance is the first luminance subtracting the attenuation luminance at the projection distance.
A rendering sub-module 850 is then used to render the valid pixel. The rendering sub-module 850 is configured to render the valid pixel using the first color information and the second luminance information.
A neighboring valid pixel sub-module 960 is configured to acquire first color information and second luminance information for a neighboring valid pixel adjacent to the valid pixel. Here, the neighboring valid pixels is, for example, eight valid pixels adjacent to said valid pixel, or four valid pixels, upper, lower, left and right adjacent to said valid pixel.
The first color information and the second luminance information of the neighboring valid pixel are then forwarded to a color and luminance adjustment sub-module 970, which is configured to adjust the first color information and the second luminance information according to the color information and the luminance information for the neighboring valid pixel. The adjustment here may be, for example, an interpolation operation based on the color information and the luminance information of the neighboring valid pixels, and the calculated color information and luminance information may serve as the adjusted first color information and second luminance information of the valid pixel, or the color information and luminance information of the pixel closest to the average color value of the neighboring valid pixels may serve as the adjusted first color information and second luminance information of the valid pixel.
The apparatus 700 shown in
Alternatively, the apparatus 700 shown in
Alternatively, the apparatus according to an embodiment of the present disclosure may include an apparatus 700 and at least one flat display panel as shown in
Alternatively, as shown in
Alternatively, the at least one processor 810 further executes the program for: determining one or more pixels in the pixel grid that overlap a projection of the valid pixel on the curved surface with respect to the visual center point; determining an overlapping area of a projection with each of the one or more pixels and a projection distance from the valid pixel to each of the one or more pixels; acquiring first color information and first luminance information of the pixel in the one or more pixels that has the largest overlapping area with the projection; determining second luminance information based on the first luminance information and the projection distance; rendering the valid pixel using the first color information and the second luminance information; acquiring first color information and the second luminance information of a neighboring valid pixel adjacent to the valid pixel; adjusting the first color information and the second luminance information according to the color information and the luminance information for the neighboring valid pixel; rendering pixels other than the valid pixel set in the flat display panel to coincide with color or hue of a bezel of the planar image display device, or wherein, the flat display panel is a transparent display panel, the at least one processor 810 further executes the program for maintaining pixels other than the valid pixel set in the flat display panel in a transparent state.
Alternatively, the apparatus further comprises: a detector 830 configured to track eye movement of the viewer; the at least one processor 810 is coupled to the detector and further executes the program for analyzing a position of the viewer's eye based on an image received from the detector to determine the visual center point.
As shown in
The functions of the respective elements shown in the figures, including any functional blocks described as “interfaces” or “engines”, may be provided by using dedicated hardware and hardware capable of executing software in conjunction with appropriate software. When provided by the processor, the functions may be provided by a single dedicated processor, provided by a single shared processor or provided by a plurality of separate processors, some of which may be shared. In addition, the explicit use of the term “processor” or “engine” should not be construed as an exclusive reference to hardware capable of executing software, but may implicitly include, but is not limited to, digital signal processor (DSP) hardware, a network processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a read only memory (ROM) for storing software, a random access memory (RAM), and a nonvolatile storage device. It can also include other conventional and/or customized hardware. Similarly, any of the switches shown in the figures is merely conceptual. Its function can be implemented by the operation of program logic, implemented by dedicated logic, implemented by the interaction between program control and dedicated logic, or even implemented manually, and it can be more specifically understood from the specific situation that the specific technology may be selected by the implementer.
It will be readily appreciated by those skilled in the art that the various method steps described above may be performed by a programmed computer. Herein, some embodiments are intended to encompass a program storage device, such as a digital data storage medium, which is a machine or computer readable and encoded machine executable or computer executable instruction program, which instructs to perform some or all of the method steps described herein. The program storage device may be, for example, a digital memory, a magnetic storage medium such as a magnetic disk or a magnetic tape, a hard disk drive, or an optically readable digital data storage medium. Embodiments are also intended to encompass a computer programmed to perform the method steps described herein.
The present disclosure is described herein by way of several exemplary embodiments. These embodiments are intended to illustrate and not limit the present disclosure. It will be apparent to those skilled in the art that the features described in one embodiment can be freely combined with the features described in other embodiments in order to obtain the described effects and/or advantages of the features.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2016 1 0345912 | May 2016 | CN | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2017/076464 | 3/13/2017 | WO | 00 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2017/202110 | 11/30/2017 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 4656521 | Trzeciak | Apr 1987 | A |
| 6462769 | Trowbridge | Oct 2002 | B1 |
| 10171768 | Rakshit | Jan 2019 | B2 |
| 20070009862 | Quinn | Jan 2007 | A1 |
| 20090201430 | Streid | Aug 2009 | A1 |
| 20120299817 | Atkins | Nov 2012 | A1 |
| 20160054648 | Read | Feb 2016 | A1 |
| 20160100119 | Lee | Apr 2016 | A1 |
| 20160133169 | Park | May 2016 | A1 |
| 20160163093 | Jeong | Jun 2016 | A1 |
| 20160187745 | Jin | Jun 2016 | A1 |
| 20160196794 | Kim | Jul 2016 | A1 |
| 20160249471 | Li | Aug 2016 | A1 |
| 20160379345 | Jin | Dec 2016 | A1 |
| 20160379601 | Jin | Dec 2016 | A1 |
| 20170171534 | Kondiparthi | Jun 2017 | A1 |
| 20170262961 | Ekambaram | Sep 2017 | A1 |
| 20180108293 | Hu et al. | Apr 2018 | A1 |
| 20180203657 | Chen | Jul 2018 | A1 |
| 20180213215 | Picarougne | Jul 2018 | A1 |
| Number | Date | Country |
|---|---|---|
| 104581114 | Apr 2015 | CN |
| 105225609 | Jan 2016 | CN |
| 106055293 | Oct 2016 | CN |
| 1326435 | Jul 2003 | EP |
| Entry |
|---|
| International Search Report from corresponding PCT Application No. PCT/CN2017/076464 dated Jun. 15, 2017 (5 pages). |
| Written Opinion from corresponding PCT Application No. PCT/CN2017/076464 dated Jun. 15, 2017 (5 pages). |
| Office Action from Chinese Application No. 201610345912.1 dated Jul. 3, 2018 (8 pages). |
| Number | Date | Country | |
|---|---|---|---|
| 20180108293 A1 | Apr 2018 | US |
