Patent Application
20060007224
This application is based upon and claims the benefit of priority from Japanese Patent Applications No. 2004-162165 filed on May 31, 2004 and No. 2005-32026 filed on Feb. 8, 2005; the entire contents of which are incorporated herein by reference.
1. Field of the Invention
The present invention relates to an image capturing function-equipped display device which includes a light-sensing element for each picture element and in which information can be inputted from a screen by means of light.
2. Description of the Related Art
As a display device which includes a light-sensing element for each picture element and which is equipped with the function of capturing an image by detecting light inputted from a screen using each light-sensing element, for example, a technology described in Japanese Unexamined Patent Publication No. 2004-93894 has been known.
In a display device of this type, when a human finger comes close to the screen, light from the screen which is reflected by the finger is received by light-sensing elements, and currents according to the amount of light received are allowed to flow therethrough. By sensing these currents, a captured image is obtained in which a region on the screen where the finger is located can be recognized.
However, in a known display device, all light-sensing elements have a single sensitivity. Accordingly, there has been a problem that an image cannot be read in any one of the cases of weak outside light and strong outside light.
For example, in a case where high-sensitivity sensors are used, a display pattern on the screen is reflected by the finger to be inputted into the light sensors in weak outside light. Accordingly, the display pattern can be obtained as a captured image. On the other hand, in strong outside light, the outside light (multiple reflection of light at the interfaces of a glass substrate, a polarizing plate, and the like) enters a space between the finger and the screen. Thus, a whole captured image appears white because the sensors have high sensitivities.
An object of the present invention is to provide an image capturing function-equipped display device in which information input can be realized by means of light in both the cases of strong outside light and weak outside light.
An image capturing function-equipped display device according to the present invention includes a picture element region including a plurality of picture elements; and a light-sensing element provided for each picture element. Here, two or more types of light-sensing elements having different optical sensitivities are arranged in the picture element region.
In the present invention, two or more types of light-sensing elements having different optical sensitivities are regularly arranged in the picture element region. By doing so, optical information can be inputted using higher-sensitivity light-sensing elements in the case of weak outside light, and optical information can be inputted using lower-sensitivity light-sensing elements in the case of strong outside light.
When the light-sensing elements are regularly arranged in the picture element region, the light-sensing elements are preferably arranged such that the sensitivities thereof differ between adjacent rows or columns. Further, the light-sensing elements may be arranged such that the sensitivities are varied in a checkerboard pattern.
Here, it is preferable that a plurality of light-sensing elements having different sensitivities are arranged in the picture element region to constitute a magic square. In this case, it is preferable that an average of values read from the plurality of light-sensing elements is regarded as a read intensity value of a picture element of interest which is contained in the magic square.
Further, it is preferable that the plurality of light-sensing elements having different sensitivities are arranged in the picture element region to constitute a magic square with alternate lines. The alternate lines are preferably any of alternate horizontal lines, alternate vertical lines, and alternate horizontal lines and alternate vertical lines.
Each light-sensing element is, for example, a gate-controlled diode including a p region, an i region, and an n region. Each low-sensitivity light-sensing element has, for example, a constitution in which p+, p−, n−, and n+ regions are arranged in this order; each high-sensitivity light-sensing element has, for example, a constitution in which p+, p−, and n+ regions are arranged in this order. In this case, in the low-sensitivity light-sensing element, the p− and n− regions correspond to the i region; meanwhile, in the high-sensitivity light-sensing element, the p− region corresponds to the i region. Here, the p+region is a region containing a high concentration of p-type impurities, and the p− region is a region containing a low concentration of p-type impurities. Similarly, the n+region is a region containing a high concentration of n-type impurities, and the n− region is a region containing a low concentration of n-type impurities.
As shown in the cross-sectional view of
Light 11 outputted by the backlight 7 is outputted to the outside of the display device through the polarizing plate 6, the counter substrate 2, the liquid crystal layer 3, the array substrate 1, and the polarizing plate 5. When a human finger 10 comes close to the outer surface of the polarizing plate 5, the light 11 is reflected by the finger 10. The light 11 reflected by the finger 10 is received by the light-sensing elements 4. Each light-sensing element 4 allows a current according to the amount of light received to flow therethrough. The image capturing function-equipped display device senses this current to obtain a captured image in which the region on the screen where the finger is located can be recognized.
Next, the operation of the image capturing function-equipped display device will be described. As shown in
On the other hand,
Thus, in this embodiment, two or more types of light-sensing elements having different optical sensitivities are regularly arranged in the picture element region. Accordingly, in a case where outside light is weak, a captured image can be obtained which is created by inputting optical information using the higher-sensitivity light-sensing elements. Meanwhile, in a case where outside light is strong, a captured image can be obtained which is created by inputting optical information using the lower-sensitivity light-sensing elements. Consequently, in both the cases of strong outside light and weak outside light, optical information input can be realized.
In this embodiment, the light-sensing elements are arranged in the picture element region such that the sensitivities differ between adjacent rows. However, the present invention is not limited to this. Various modifications will be described below.
As shown in the plan view of
As shown in the plan view of
As shown in the plan view of
Incidentally, light-sensing elements having three or more types of different sensitivities may be arranged such that the sensitivities differ between adjacent rows or columns, or may be arranged in a checkerboard pattern.
Moreover, the sensitivity of the light-sensing element can be adjusted by changing the voltage on the gate electrode, if the light-sensing element is a gate-controlled diode. Further, the sensitivity of the light-sensing element can also be adjusted by changing at least one of the width and length of the light-sensing element.
As shown in the plan view of
In the case of the above-described arrangement, signals read by the sensors are processed in an external signal processing unit (not shown) as follows. First, an average of values (each 0 or 1) read from nine picture elements, including a picture element of interest and the surrounding ones, is regarded as the intensity value of the picture element of interest at the center of the three-by-three picture element region. This is performed on all picture elements. Thus, a new multi-level image is obtained. In the multi-level image thus obtained, it is unlikely that a portion indicated by a finger or the like is saturated and entirely becomes white or black in various ambient lights. This increases a probability that a read can be reliably performed. By performing predetermined image processing on this image, an accurate operation can be performed. For example, a coordinate detection operation or the like is performed based on this multi-level image.
As shown in the plan view of
Next, the arrangement of the light-sensing elements in which a consideration is given to the drive polarities of the picture elements will be described. Here, suppose that the horizontal lines of the picture elements having positive drive polarity and those of the picture elements having negative drive polarity are alternately arranged.
The polarity distribution diagram of
In light of this, in an image capturing function-equipped display device of a seventh embodiment, a plurality of light-sensing elements having different sensitivities are arranged such that a magic square is constituted with alternate horizontal lines and with alternate vertical lines. Here, as an example, as shown in the plan view of
As for the picture element region 41 in this drawing, the picture elements of which numbers are surrounded by circles in the drawing correspond to the three-by-three picture elements with alternate horizontal lines and with alternate vertical lines. The polarities of these picture elements are positive in common. Incidentally, each number in the drawing indicates the sensitivity of the light-sensing element. The fact that the value of a photocurrent flowing through the light-sensing element in constant light increases in proportion to the number is the same as in the aforementioned embodiments.
When finding the multi-level value of the picture element of interest at the center of the picture element region 41, an average is taken over the intensity values of the nine picture elements of which numbers are surrounded by circles. Since all the intensity values of these picture elements are positive polarity, a correct multi-level value can be obtained.
As for the picture element region 42 in the drawing, all the polarities of the three-by-three picture elements with alternate horizontal lines and with alternate vertical lines, which picture elements have numbers surrounded by circles, are negative. Accordingly, when finding the multi-level value of the picture element of interest at the center, a correct multi-level value can be obtained by taking an average over the intensity values of these picture elements. Although the case of the three-by-three picture elements has been described in this embodiment, four-by-four picture elements or eight-by-eight picture elements may be adopted. Considering the internal constitution (portion for calculating one intensity value using the values of the picture elements in a predetermined range) of an IC for the sensor, in a case where a magic square is constituted by the four-by-four picture elements and where the predetermined range corresponds to 16-by-16 picture elements, the memory of the IC can be efficiently configured. This is because, in many cases, the memory of the IC is arranged and configured such that eight bits constitute one character.
Thus, this embodiment makes it possible to exclude the influence of drive polarity and to obtain correct multi-level values in both of picture elements having positive polarity and those having negative polarity.
According to the above-described embodiments, since the plurality of light-sensing elements having different sensitivities are arranged, high-sensitivity sensors respond in a dark environment, and low-sensitivity sensors respond in a bright environment. Consequently, multi-level values having a wide dynamic range can be obtained. Further, since light-sensing elements having sensitivities appropriate to ambient light respond, image-capturing time can be shortened. Consequently, the number of captured-image frames per unit time can be increased.
A liquid crystal display device (LCD) for a mobile phone is often used in combination with a transparent acrylic plate as a protective plate. In this case, the finger does not directly touch a liquid crystal cell but touches the surface of the protective plate. Accordingly, the light sensors incorporated in the liquid crystal cell sense and respond to light due to multiple reflection of light (stray light) between the protective plate and the liquid crystal cell, between the glass-liquid crystal interface of the liquid crystal cell and the glass-polarizing plate interface thereof, between a backlight surface and the glass-polarizing plate interface, and the like, even if the light sensors lie under the finger. Consequently, in the case of a simple binary read in which “a white portion in the result of the read means outside light” and in which “a black portion means the finger,” white saturation occurs and the finger cannot be recognized in strong outside light due to white saturation. It is difficult to obtain a high S/N ratio because the finger itself does not have a light source. There occurs the problem that the shadow of the finger cannot be distinguished from the background (white saturation occurs) in a binary read in which a specific illumination is set as a threshold and in which a read process is performed by regarding values above the threshold as white and regarding values below the threshold as black (this problem is the same in all the aforementioned embodiments to greater or lesser degrees because of the thickness of the glass substrate even though the protective plate does not exist).
Accordingly, a constitution for reading the difference between the intensity of the finger and that of the background is needed. It is possible to conceive of performing area coverage modulation on raw data (binary) read. Further, an anti-white saturation measure is taken by increasing the number of the levels of the sensors. Here, the area coverage modulation means calculating an average value of binary outputs of the plurality of sensors in the vicinity of a picture element of interest and regarding the average as a new intensity value. The size of the vicinity can be optimized based on the size of an indicating substance such as a finger, the pitch of the sensors, and the like. The phrase “increasing the number of the levels of the sensors” means the following: insensitive sensors having a plurality of levels are intentionally mixed in addition to relatively sensitive sensors which are effective in dark places, and the sensors are made to function in a wider illumination range, thus preventing white saturation. Also from such a viewpoint, the aforementioned embodiments are effective. Further, picture elements containing the sensors having the plurality of levels have slightly different shapes. If these picture elements are regularly arranged, periodic display unevenness is prone to be observed when normal display is performed. Further, there are cases where periodic unevenness occurs in a captured image. Accordingly, the plurality of picture elements having the plurality of sensors are preferably irregularly arranged. The aforementioned magic square arrangements are the examples thereof. In the aforementioned examples, descriptions have been made by setting the levels of the sensors as 1:2: . . . :9. However, precisely equal differences are not needed. Moreover, equal ratios may be adopted. The number of the levels has been nine, but is not limited to it. It is essential that the number of the sensors which respond to the illumination of outside light increases. If there are portions in which the number of the sensors responding to the illumination of outside light does not increase, that is a problem. This is because the difference between the intensity of outside light and that of a finger cannot be read in the relevant region.
A similar thing can also be performed by reading as multi-level signals the outputs of the sensors on the glass substrate from the first use of multi-level AID converter. However, the constitution (binary sensor signals are outputted from the glass substrate and subjected to area coverage modulation on the outside to be converted into multi-level data) of the present invention is more advantageous in terms of cost and ease of design (non-severe noise design).
The method of changing the sensitivity of the sensor is variously devised besides the size of the sensor is changed. For instance, it is possible to change the exposure time of each line and to take picture. It is preferable to combine changing the size of the sensor and changing the exposure time.
| Number | Date | Country | Kind |
|---|---|---|---|
| P2004-162165 | May 2004 | JP | national |
| P2005-32026 | Feb 2005 | JP | national |
