Korean Patent Application No. 10-2012-0127253, filed on Nov. 12, 2012, and entitled: “Method and Apparatus for Interpolating Color,” which is incorporated by reference herein in its entirety.
1. Field
Embodiments herein relate to display devices.
2. Description of the Related Art
An image sensor converts an optical image signal into an electrical image signal. To produce a color image, pixels which emit light of different may be included in the image sensor. When a driving signal of a color (e.g., a red pixel value, green pixel value, or blue pixel value) is missing at each pixel location, a digital imaging process for estimating the missing pixel value may be performed. Examples of a digital imaging process of this type include a demosaicing algorithm and an interpolation algorithm. However, these and other digital imaging processes have proven to have drawbacks, not the least of which includes computational complexity.
Embodiments are directed to a method for generating an image by interpolating one or more pixel values of a predetermined color.
In accordance with one embodiment, a method for interpolating a color includes interpolating, in a first pixel of a first color, a first pixel value based on pixel values of pixels adjacent to the first pixel; skipping an interpolation on a second pixel of a second color corresponding to the first pixel of the first color; and generating an image based on the interpolated first pixel value and an uninterpolated pixel value of the second pixel. The first pixel may be included in a first layer, the second pixel may be included in a second layer, and the first layer and the second layer may be vertically stacked.
Also, the first layer may have a pattern of pixels of the first color and a third color, and the second layer may include pixels of the second color corresponding to the pixels the pattern of the first layer. The pattern may include an alternating pattern of pixels of the first color and the third color. The first layer may be over the second layer, or the second layer may be over the first layer. The first interpolated pixel value in the first pixel may computed by Bilinear interpolation, by Constant Hue base interpolation, or by edge sensing interpolation.
In accordance with another embodiment, a device includes a pixel array including a first layer having a first pixel and a second layer having a second pixel; and an image signal processor which interpolates a first pixel value in the first pixel based on pixel values of pixels adjacent to the first pixel output from the pixel array, which skips interpolation of the second pixel, and which generates an image based on the interpolated first pixel value and an uninterpolated pixel value of the second pixel.
Also, the first layer and the second layer may be vertically stacked. The first layer may include a third pixel, and the first pixel and the third pixel are disposed in a predetermined pattern. The predetermined pattern may be an alternating pattern of first and third pixels.
Also, the first interpolated pixel value in the first pixel may be computed by Bilinear interpolation, by Constant Hue based interpolation, or by edge sensing interpolation. The first pixel may be a blue pixel or a red pixel, and the second pixel may be a green pixel. The second layer may be formed of an organic photoelectric-conversion film.
Also, the first layer has a pattern of pixels of the first color and a third color, and the second layer includes pixels of the second color corresponding to the pixels the pattern of the first layer, wherein the pattern of pixels includes an alternating pattern of pixels of the first and third colors.
Features will become apparent to those of ordinary skill in the art by describing in detail exemplary embodiments with reference to the attached drawings in which:
Example embodiments will now be described more fully hereinafter with reference to the accompanying drawings; however, they may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey exemplary implementations to those skilled in the art.
In the drawing figures, the dimensions of layers and regions may be exaggerated for clarity of illustration. It will also be understood that when a layer or element is referred to as being “on” another layer or substrate, it can be directly on the other layer or substrate, or intervening layers may also be present. Further, it will be understood that when a layer is referred to as being “under” another layer, it can be directly under, and one or more intervening layers may also be present. In addition, it will also be understood that when a layer is referred to as being “between” two layers, it can be the only layer between the two layers, or one or more intervening layers may also be present. Like reference numerals refer to like elements throughout.
The microlens 11 collects light incident from an external source. In an alternative embodiment, the pixel array 10 may be embodied without the microlens 11. The first layer 20 and the second layer 30 will be described in detail referring to
A photo-detector 15 generates photoelectrons in response to light incident from an external source. The photo-detector 15 is formed in the epitaxial layer 13. The photo-detector 15 may be formed by or include a photodiode, a phototransistor, a photogate, or a pinned photodiode (PPD) as a photosensitive element.
The inter-metal dielectric layer 17 may be formed of an oxide layer or a composite layer of an oxide layer and a nitride layer. The oxide layer may be a silicon oxide layer. The inter-metal dielectric layer 17 may include metal patterns 19.
Electrical wiring required for a sensing operation of the pixel array may be formed of the metal patterns 19. In addition, according to an example embodiment, the metal patterns 19 may be used to reflect light incident through the photo-detector 15 back to the photo-detector 15. The metal patterns 19 may be copper, titanium, or titanium nitride. The substrate 21 may be a silicon substrate.
The second layer 30 includes red pixels R and blue pixels B disposed in a predetermined pattern. In the example, shown the predetermined pattern is a checker pattern. Although, the red and blue pixels may be disposed in a different pattern.
Also, as shown in
Herein, the term ‘pixel’ may be understood to correspond to a component unit generating a color pixel signal having a color pixel value. For example, a green pixel G denotes a component unit generating a green pixel signal having a green pixel value corresponding to wavelengths belonging to a green region of visible light spectrum. A red pixel R denotes a component unit generating a red pixel signal having a red pixel value corresponding to wavelengths belonging to a red region of the visible light spectrum. A blue pixel B denotes a component unit generating a blue pixel signal having a blue pixel value corresponding to wavelengths belonging to a blue region of the visible light spectrum.
The green pixel G may absorb wavelengths of the green region of the visible light spectrum and generate a green pixel signal having a green pixel value corresponding to wavelengths belonging to the green region. That is, the green pixel G converts visible light including wavelengths of the green region into a green pixel signal. Wavelengths of the remaining regions, except for wavelengths belonging to the green region and absorbed by the green pixel G in the visible light, pass through the second layer 30.
The red pixel R may include a yellow organic color filter YF and the photo-detector 15. The yellow organic color filter YF absorbs wavelengths of a blue region among wavelengths of the remaining region, so as to remove wavelengths belonging to the blue region from wavelengths of the remaining regions. The yellow organic color filter YF does not remove wavelengths belonging to the green region to be absorbed by the green pixel G in the visible light spectrum.
The photo-detector 15 converts wavelengths of the visible light, passing through the red pixel R, into a red pixel signal. That is, the red pixel R generates a red pixel signal having a red pixel value using the photo-detector 15.
A blue pixel B may include a cyan organic color filter CF and photo-detector 15.
The cyan organic color filter CF absorbs wavelengths of the red region, so as to remove wavelengths belonging to the red region from wavelengths of the remaining regions, but does not remove wavelengths belonging to a green region absorbed by a green pixel G in the visible light spectrum.
The photo-detector 15 converts wavelengths of the visible light, passing through the blue pixel B, into a blue pixel signal. That is, the blue pixel B generates a blue pixel signal having a blue pixel value using the photo-detector 15.
The image sensing system 1 may sense an object 400 imaged through a lens 500 by control of the digital signal processor 200. The digital signal processor 200 may output a color image sensed and output by the image sensor 100 to a display unit 300. The display unit 300 may be any display device capable of outputting an image. For example, the display unit 300 may be one included in or coupled to a computer, a cellular phone, or another type of image output terminal.
The digital signal processor 200 may include a camera controller 210, an image signal processor 220, and an interface (I/F) 230. The camera controller 210 controls a control register block 175. The camera controller 210 may control the image sensor 100, i.e., the control register block 175, using an Inter-Integrated Circuit (I2C); however embodiments are not restricted thereto.
The image signal processor (ISP) 220 receives digital pixel signals output from a buffer 190, processes the received digital pixel signals to be easily visible to people, and outputs the processed image to the display unit 300 through the PC I/F 230. For example, the ISP 220 may perform an interpolation operation using digital pixel signals output from the image sensor 100.
In
The image sensor 100 includes the pixel array 10 illustrated in
The pixel array 10 may include pixels in a matrix form connected with a plurality of row lines and a plurality of column lines, respectively.
The timing generator 165 may control an operation of the row driver 120 and the ADC 130 by outputting control signals to each of the row driver 120 and the ADC 130. The control register block 175 may control each operation of the timing generator 165 and the buffer 190 by outputting control signals to each of the timing generator 165 and the buffer 190. Here, the control register block 175 operates based on the control of the camera controller 210. The camera controller 210 may be embodied in hardware or software.
The row driver 120 may drive the pixel array 10 row by row. For example, the row driver 120 may generate a row selection signal. That is, the row driver 120 may decode a row control signal, e.g., an address signal, generated by the timing generator 165, and select at least one row line from row lines included in the pixel array 10 in response to the decoded row control signal. In addition, the pixel array 10 outputs pixel signals from a row, selected by a row selection signal provided from the row driver 120, to the ADC 130.
The ADC 130 converts pixel signals output from the pixel array 10 into digital pixel signals and outputs the digital pixel signals to the buffer 190.
The pixel signals 40 are output from the blue pixels B and the red pixels R in the second layer 30. The locations of the pixel signals 40 correspond to respective locations of pixels B or R of the second layer 30. Blue pixel signals B12, B14, B21, B23, B32, B34, B41, and B43 indicate blue pixel values of corresponding ones of the blue pixel signals output from the blue pixels B of the second layer 30. Red pixel signals R11, R13, R22, R24, R31, R33, R42, and R44 indicates red pixel values of corresponding ones of the red pixel signals output from red pixels R of the second layer 30.
Green pixel signals 50 are output from the green pixels G of the first layer 20. The locations of the green pixel signals 50 correspond to respective locations of green pixel G of the first layer 20. Green pixel signals G11, G12, G13, G14, G21, G22, G23, G24, G31, G32, G33, G34, G41, G42, G43, and G44 indicate green pixel values of corresponding ones of the green pixel signals output from the green pixels G of the first layer 20. The image sensor 100 outputs green pixel signals 50 per green pixel G, so that demosaic processing is not required on each green pixel G.
The numbers of blue pixels B and red pixels may be different from the number of green pixels. For example, in accordance with one embodiment, the number of blue pixels B and the number of red pixels R, which are arranged on the second layer 30, may be half of the number of green pixels G arranged on the first layer 20. Accordingly, each of the number of blue pixel signals and the number of red pixel signals are a half of the number of green pixel signals, respectively.
As shown in
Similarly, a red pixel signal having a red pixel value R22 is output from the red pixel 43. However, a blue pixel signal is not output from the red pixel 43. Accordingly, a method for interpolating a blue pixel value in the red pixel 43 is needed. That is, demosaic processing may be performed for interpolating a blue pixel value in the red pixel 43. Demosaic processing may be performed, for example, by the image signal processor 220.
Red pixel signals 60 include red pixel signals output from the red pixels R and red pixel signals having red pixel values interpolated obtained after the demosaic processing, in block 55. The symbols r12, r14, r21, r23, r32, r34, r41, and r43 indicate interpolated red pixel values.
Blue pixel signals 70 include blue pixel signals output from the blue pixels B and blue pixel signals having the interpolated blue pixel values obtained after the demosaic processing in block 55. The symbols b11, b13, b22, b24, b31, b33, b42, and b44 indicate interpolated blue pixel values. Each interpolated red pixel value and/or each interpolated blue pixel value may be computed by using pixel values of adjacent pixels.
In accordance with one embodiment, the interpolated blue pixel values may be computed based on Equation 1:
b
xy=(B(x−1)y+Bx(y−1)+Bx(y+1)+B(x+1)y))/4 (1)
In Equation 1, the symbol bxy indicates an interpolated blue pixel value in a red pixel, x indicates a row, y indicates a column, symbols B(x−1)y, Bx(y−1), Bx(y+1), and B(x+1)y indicate blue pixel values of adjacent blue pixels to the red pixel. For example, the interpolated blue pixel value b22 may be computed based on blue pixel values B12, B21, B23, and B32 of adjacent blue pixels 73, 75, 77, and 79 to a red pixel 71. A location of the red pixel 71 corresponds to a location of the red pixel 43. When the blue pixel value b22 is interpolated using the Equation 1, Bilinear interpolation may be defined.
In accordance with the same or another embodiment, a blue pixel value may be interpolated based on Equation 2.
b
xy
=G
xy+(Bavg−Gavg) (2)
In accordance with this same or another embodiment, Bavg and Gavg may be computed based on Equations 3 and 4:
B
avg=(B(x−1)y+Bx(y−1)+Bx(y+1)+B(x+1)y)/4 (3)
G
avg=(B(x−1)y+Gx(y−1)+Gx(y+1)+G(x+1)y)/4 (4)
The symbol bxy indicates the interpolated blue pixel value in a red pixel, x indicates a row, y indicates a column, symbols B(x−1)y, Bx(y−1), Bx(y+1), and B(x+1)y indicate pixel values of adjacent pixels to the red pixel, Gxy indicates a green pixel value of a green pixel corresponding to the red pixel, and each of G(x−1)y, Gx(y−1), Gx(y+1), and G(x+1)y indicate pixel values of adjacent pixels to the green pixel.
For example, the interpolated blue pixel value b22 may be computed using blue pixel values B12, B21, B23, and B32 of adjacent blue pixels 73, 75, 77, and 79 to the red pixel 71 and green pixel values G22, G12, G21, G23, and G32 of green pixels. When the blue pixel value b22 is interpolated based on Equations 2 to 4, Constant Hue based interpolation may be defined.
In accordance with another embodiment, a blue pixel value may be interpolated based on Equations 5, 6, and 7.
When GH>GV+TH, bxy=Gxy+(Bavg, V−Gavg, V) (5)
When GH<GV+TH, and GV>GH+TH, bxy=Gxy+(Bavg, H−Gavg, H) (6)
When GH<GV+TH, and GV<GH+TH, bxy=Gxy+(Bavg−Gavg) (7)
The GV, the GH, the Bavg, V, the Bavg, H, the Gavg, the Gavg, H, the Bavg and the Gavg may be computed based on Equations 8 to 15:
G
V
=|G(x−1)y−G(X−1)(y−1)|+|Gxy−Gx(y−1)|+|G(x+1)y−G(x+1)(y−1)| (8)
G
H
=|G
(x−1)(y−1)
−G
x(y−1)|
+|G
x(y+1)
−G
xy
|+|G
(x+1)(y+1)
−G
(x+1)y| (9)
B
avg, V=(B(x−1)y+B(x+1)y)/2 (10)
B
avg, H=(Bx(y−1)+Bx(y+1))/2 (11)
G
avg, V=(G(x−1)y+G(x+1)y)/2 (12)
G
avg, H=(Gx(y−1)+Gx(y+1))/2 (13)
B
avg=(B(x−1)y+Bx(y−1)+Bx(y+1)+B(x+1)y)/4 (14)
G
avg=(G(x−1)y+Gx(y−1)+Gx(y+1)+G(x+1)y)/4 (15)
The symbol bxy indicates the first interpolated pixel value in the first pixel, x indicates a row, y indicates a column, each of the B(x−1)y, Bx(y−1), Bx(y+1), and B(x+1)y indicates a pixel value of adjacent pixels to the first pixel, Gxy indicates the second pixel value of the second pixel corresponding to the first pixel, each of the G(x−1)(y−1), G(x−1)y, G(x−1)(y+1), Gx(y−1), Gx(y+1), G(x+1)(y−1), G(x+1)y, and G(x+1)(y+1) indicates a pixel value of adjacent pixels to the second pixel, and the TH indicates a threshold value.
When the first interpolated pixel value bxy is interpolated based on Equations 5 to 15, edge sensing interpolation may be defined. Similarly, the interpolated red pixel value may be computed using pixel values of adjacent pixels.
A CSI host 1012 embodied in the application processor 1010 may perform a serial communication with a CSI device 1041 of the image sensor 1040 through a camera serial interface (CSI). Here, for example, a deserializer (DES) may be embodied in the CSI host 1012, and a serializer (SER) may be embodied in the CSI device 1041. The image sensor 1040 indicates the image sensor 100 described in
A DSI host 1011 embodied in the application processor 1010 may perform a serial communication with a DSI device 1051 of the display 1050 through a display serial interface (DSI). Here, for example, a serializer (SER) may be embodied in the DSI host 1011, and a deserializer (DES) may be embodied in the DSI device 1051.
The image sensing system 1000 may further include a RF chip 1060 which may communicate with the application processor 1010. A PHY 1013 of the image sensing system 1000 may transmit or receive data to/from a PHY 1061 of the RF chip 1060 according to MIPI DigRF.
The image sensing system 1000 may further include a GPS receiver 1020, a storage 1070, a mike 1080, a DRAM 1085, and a speaker 1090. The image sensing system 1000 may communicate using a Wimax 1030, a WLAN 1100, and a UWB 1110.
A method and a device for interpolating a color according to the aforementioned embodiments may decrease computational complexity of a demosaicing algorithm and hardware resources for executing the demosaicing algorithm by suggesting the demosaicing algorithm.
Example embodiments have been disclosed herein, and although specific terms are employed, they are used and are to be interpreted in a generic and descriptive sense only and not for purpose of limitation. In some instances, as would be apparent to one of ordinary skill in the art as of the filing of the present application, features, characteristics, and/or elements described in connection with a particular embodiment may be used singly or in combination with features, characteristics, and/or elements described in connection with other embodiments unless otherwise specifically indicated. Accordingly, it will be understood by those of skill in the art that various changes in form and details may be made without departing from the spirit and scope of the present invention as set forth in the following claims.
Number | Date | Country | Kind |
---|---|---|---|
10-2012-0127253 | Nov 2012 | KR | national |