The disclosure relates to an image-sensing device.
With advances being made in technology, electronic devices equipped with a camera have become very popular. However, a modular lens in a conventional camera, a.k.a. a color image sensing (CIS) device, is generally an essential component for capturing incoming light and converting this captured light into digital images. However, due to the limitations of conventional imaging techniques, an image is formed with a lens, and a modular lens takes up a large portion of the available space within the camera. Since the size of portable electronic devices has become smaller and smaller, a large-sized modular lens is not appropriate for these devices.
Accordingly, there is demand for a lens-free image sensor to reduce the size of the camera.
The disclosure provides an image-sensing device, which includes a plurality of photoelectric elements for receiving incident light. The photoelectric elements are arranged into a plurality of unit cells, and each of the unit cells includes a first photoelectric element, a second photoelectric element, a third photoelectric element, and a fourth photoelectric element. The first photoelectric element, the second photoelectric element, the third photoelectric element, and the fourth photoelectric element in each of the unit cells are formed of pillar structures, and the first photoelectric element, the second photoelectric element, the third photoelectric element, and the fourth photoelectric element are all different sizes. The first photoelectric element in each of the unit cells captures a first image in a first phase, the second photoelectric element in each of the unit cells captures a second image in a second phase, the third photoelectric element in each of the unit cells captures a third image in a third phase, the fourth photoelectric element in each of the unit cells captures a fourth image in a fourth phase. The first phase, the second phase, the third phase, and the fourth phase are different.
The disclosure can be more fully understood by reading the subsequent detailed description and examples with references made to the accompanying drawings, wherein:
Technical terms of the disclosure are based on general definition in the technical field of the disclosure. If the disclosure describes or explains one or some terms, definition of the terms is based on the description or explanation of the disclosure. Each of the disclosed embodiments has one or more technical features. In possible implementation, a person skilled in the art would selectively implement all or some technical features of any embodiment of the disclosure or selectively combine all or some technical features of the embodiments of the disclosure.
In each of the following embodiments, the same reference number represents the same or similar element or component.
For example, the initial phase of the reference wave is zero and changes by π/2 at each step. Assuming a 4-step phase-shifting digital holography algorithm is used and the intensity of the interference patterns at different phases such as 0, π/2, π, and 3π/2 may be respectively expressed in the formulas (1)-(4):
I0=|ψ0|2+|ψr|2+ψ0ψr*+ψ0*ψr (1)
Iπ/2=|ψ0|2+|ψr|2+jψ0ψr*−jψ0*ψr (2)
Iπ=|ψ0|2+|ψr|2+ψ0ψr*−ψ0*ψr (3)
I3π/2=|ψ0|2+|ψr|2−jψ0ψr*+jψ0*ψr (4)
After obtaining the intensity of the interference patterns at phases 0, π/2, π, and 3π/2, the complex amplitude of the object light is given by the following formula (5):
The complex amplitude of the object light is sometimes referred to as the complex hologram image because we may retrieve the amplitude distribution of the object light in the object plane from ψ0 by performing digital back-propagation.
Accordingly, the complex amplitude of the reference light must be known in order to calculate object waves. Usually, the reference light is a plane wave or a spherical wave and therefore its phase is known without any measurement. One having ordinary skill in the art will appreciate the techniques for reconstructing an object image using object waves at different phases, and thus the details will be omitted here.
It should be noted that the phase-shifting digital holography device 100 described in the example of
However, the concept of phase delay of the phase-shifting digital holography algorithm may be used in an image-sending device of the disclosure.
In the sensor array 210, the photoelectric elements are arranged into a plurality of unit cells 220, 230, 240 and 250. For example, each of unit cells 220, 230, 240 and 250 includes a photoelectric element 221, a photoelectric element 222, a photoelectric element 223 and a photoelectric element 224, and the photoelectric element 221, the photoelectric element 222, the photoelectric element 223 and the photoelectric element 224 are arranged into a 2×2 array.
Furthermore, the photoelectric element 221, the photoelectric element 222, the photoelectric element 223 and the photoelectric element 224 in each of the unit cells 220, 230, 240 and 250 are formed of pillar structures. In the embodiment, materials of the pillar structures are, for example, single crystal silicon, polycrystalline silicon (poly Si), amorphous silicon, Si3N4, GaP, TiO2, AlSb, AlAs, AlGaAs, AlGaInP, BP, ZnGeP2, any other applicable material, or a combination thereof, but the embodiment of the disclosure is not limited thereto.
In addition, the photoelectric element 221, the photoelectric element 222, the photoelectric element 223 and the photoelectric element 224 are different sizes. For example, the size of the photoelectric element 221 is less than the size of the photoelectric element 222. The size of the photoelectric element 222 is less than the size of the photoelectric element 223. The size of the photoelectric element 223 is less than the size of the photoelectric element 224.
Furthermore, the photoelectric element 221, the photoelectric element 222, the photoelectric element 223 and the photoelectric element 224 may be different diameters, and the diameter of each of the photoelectric element 221, the photoelectric element 222, the photoelectric element 223 and the photoelectric element 224 represents a specific phase of a corresponding phase-shifting hologram image.
Specifically, a 4-step phase-shifting holography method is employed into the architecture of the image-sensing device 200. For example, the diameters of the photoelectric element 221, the photoelectric element 222, the photoelectric element 223 and the photoelectric element 224 are d0, d1, d2, and d3 that correspond to the phase δ0, the phase δ1, the phase δ2, and the phase δ3, respectively. The values of the phase δ0, the phase δ1, the phase δ2, and the phase δ3 are, for example 0, π/2, π, and 3π/2, respectively. In the embodiment, a corresponding relationship of the diameters of the photoelectric elements and the phases is as shown in
It should be noted that the unit cell 220 is repeatedly arranged in the sensor array 210, and each of the photoelectric element 221, the photoelectric element 222, the photoelectric element 223 and the photoelectric element 224 in each of unit cells 220, 230, 240 and 250 may capture an image in an individual phase of four different phases. For example, the photoelectric element 221 in each of the unit cells 220, 230, 240 and 250 captures a first image in a first phase (such as the phase δ0), the photoelectric element 222 in each of the unit cells 220, 230, 240 and 250 captures a second image in a second phase (such as the phase δ1), the photoelectric element 223 in each of the unit cells 220, 230, 240 and 250 captures a third image in a third phase (such as the phase δ2), the photoelectric element 224 in each of the unit cells 220, 230, 240 and 250 captures a fourth image in a fourth phase (such as the phase δ3).
Since the first image, the second image, the third image, and the fourth image are captured by the photoelectric element 221, the photoelectric element 222, the photoelectric element 223 and the photoelectric element 224 in each of the unit cells 250, and thus the locations of the first image, the second image, the third image, and the fourth image are substantially the same. For example, the phase-shifting hologram image for the phase δ0 may be obtained from the captured image of the photoelectric element 221 in each of unit cells 220, 230, 240 and 250. Similarly, the phase-shifting hologram image for the phase δ1, the phase δ2, and the phase δ3 may be obtained from the captured image of the photoelectric element 222, the photoelectric element 223 and the photoelectric element 224 in each of the unit cells 220, 230, 240 and 250, respectively.
After obtaining phase-shifting hologram images in four phases, the object wave in the Fourier domain may be obtained using formula (5). Subsequently, an inverse Fourier transform is performed on the object wave to reconstruct the object image in the spatial domain. Alternatively, a transfer function H(x, y) for transforming the object wave in the Fourier domain to the object image in the spatial domain may be estimated in advance, and thus a convolution between the object wave and the transfer function may be performed to obtain the object image.
In the embodiment, each of the unit cells 220, 230, 240 and 250 is formed as a 2×2 array, an order of the first photoelectric element 221, the second photoelectric element 222, the third photoelectric element 223 and the fourth photoelectric element 224 in the 2×2 array is fixed, and an order of each of unit cells 220, 230, 240 and 250 is fixed, as shown in
In some embodiments, each of the unit cells 220, 230, 240 and 250 is formed as a 2×2 array, an order of the first photoelectric element 221, the second photoelectric element 222, the third photoelectric element 223 and the fourth photoelectric element 224 in the 2×2 array is flexible, and an order of each of the unit cells 220, 230, 240 and 250 is flexible, as shown in
Alternatively, in some embodiments, each of the unit cells 220, 230, 240 and 250 is formed as a 2×2 array, an order of the first photoelectric element 221, the second photoelectric element 222, the third photoelectric element 223 and the fourth photoelectric element 224 in the 2×2 array is fixed, each of the unit cells 220, 230, 240 and 250 is repeated with a predetermined degree rotation to the right, or each of the unit cells 220, 230, 240 and 250 is repeated with a flip to the right, or each of the unit cells 220, 230, 240 and 250 is repeated with the flip and the predetermined degree rotation to the right, as shown in
It can be seen in
It can be seen in
It can be seen in
Accordingly, the image-sensing devices 200 in
In step S820, the method involves calculating the object wave in the Fourier domain according to the four phase-shifting hologram images in different phases. For example, the intensities of the phase-shifting hologram images in different phases (such as 0, π/2, π, and 3π/2) may be calculated using formulas (1)˜(4), and the object wave may be calculated using formula (5). However, to simplify the calculation of the object wave, the object wave φ0 may be calculated approximately using the following formula (6):
φ0≈(I0−Iπ)−ƒ(Iπ/2−I3π/2) (6)
In step S830, the method involves reconstructing the object image according to the object wave. For example, the object wave φ0 is in the Fourier domain and the object image is in the spatial domain, and thus an inverse Fourier transform may be applied on the object wave φ0 to reconstruct the object image. Alternatively, a transfer function H(x, y) for transforming the object wave in the Fourier domain to the object image in the spatial domain may be estimated in advance, and thus a convolution between the object wave and the transfer function may be performed to obtain the object image.
The sensor array 910 includes a plurality of photoelectric elements for receiving incident light. In the sensor array 910, the photoelectric elements are arranged into a plurality of unit cells 930, 940, 950 and 960. For example, the unit cell 930 (such as a first unit cell) includes a photoelectric element 931, a photoelectric element 932, a photoelectric element 933 and a photoelectric element 934. The unit cell 940 (such as a second unit cell) includes a photoelectric element 941, a photoelectric element 942, a photoelectric element 943 and a photoelectric element 944. The unit cell 950 (such as a third unit cell) includes a photoelectric element 951, a photoelectric element 952, a photoelectric element 953 and a photoelectric element 954. The unit cell 960 (such as a fourth unit cell) includes a photoelectric element 961, a photoelectric element 962, a photoelectric element 963 and a photoelectric element 964.
In the embodiment, the photoelectric elements 931˜934, the photoelectric elements 941˜944, the photoelectric elements 951˜954 and the photoelectric elements 961˜964 may be arranged into a 2×2 array, respectively. The photoelectric elements 931˜934, the photoelectric elements 941˜944, the photoelectric elements 951˜954 and the photoelectric elements 961˜964 are equal to or similar to the photoelectric elements 221˜224 in
In addition, the unit cells 930, 940, 950 and 960 are arranged into a plurality of macro unit cells 920. For example, each of the macro unit cells 920 may include the unit cell 930, the unit cell 940, the unit cell 950 and the unit cell 960. In the embodiment, the unit cell 930, the unit cell 940, the unit cell 950 and the unit cell 960 may be arranged into a 2×2 array.
The filter array 970 is disposed on the photoelectric elements, i.e., the filter array 970 is disposed on the sensor array 910. In addition, the filter array 970 may include a plurality of color filters, such as red filters 972, green filters 974 and 976, and blue filters 978. For example, two green filters 974 and 976, one red filter 972, and one blue filter 978 are arranged into a 2×2 array. In the embodiment, the green filters 974 and 976 may extract green light from the incident light, the red filter 972 may extract red light from the incident light, and the blue filter 978 may extract blue light from the incident light.
The sensor array 910 may receive the incident light via the filter array 970. The unit cell 930 may correspond to the green filter 974 in the filter array 970, the unit cell 940 may correspond to the green filter 976 in the filter array 970, the unit cell 950 may correspond to the red filter 972 in the filter array 970, and the unit cell 960 may correspond to the blue filter 978 in the filter array 970. Thus, the unit cells 930, 940, 950, and 960 in each of the macro unit cells 920 may receive the green light, the green light, the red light and the blue light via the green filter 974, the green filter 976, the red filter 972 and the blue filter 978 in the filter array 910, respectively. Specifically, the four unit cells 930, 940, 950, and 960 in each of the macro unit cell 920 are configured to capture green, blue, red, and green images in four different phases, such as 0, π/2, π, and 3π/2.
Given that Hζr, Hζg, and Hζb represent the wavelengths of the red light, green light, and blue light respectively, it can be concluded that the relationship between the wavelengths is Hζr>Hζg>Hζb, since the red light has the longest wavelength and the blue light has the shortest wavelength among red, green, and blue lights. Accordingly, assuming that the photoelectric elements in the sensor array 910 are made of the same material, the photoelectric elements 951˜954 in the unit cell 950 for receiving the red light have relatively greater heights than the photoelectric elements in other unit cells in the macro unit cell 920. That is, the heights of the photoelectric elements in each unit cell are proportional to the wavelength of the received light.
For example, the heights of the photoelectric elements 951˜954 in the unit cell 950 (corresponding to the red filter 972) are higher than the heights of the photoelectric elements 931˜934 and 941˜944 in the unit cell 930 and the unit cell 940 (corresponding to the green filters 941 and 976), and the heights of the photoelectric element 931˜934 and 941˜944 in the 930 and the unit cell 940 (corresponding to the green filters 941 and 976) are higher than the heights of the photoelectric elements 961˜964 in the unit cell 960, as shown in
Since the macro unit cell 920 is repeatedly arranged in the sensor array 910, four phase-shifting hologram images are obtained by combining images captured by each of the unit cells 930, 940, 950, and 960 of the macro unit cells 920 in the sensor array 910, and thus total 16 phase-shifting hologram images may be obtained. It should be noted that the green phase-shifting hologram image captured by the unit cell 930 is the same as that captured by the unit cell 940. For example, the total 16 phase-shifting hologram images may be (Rδ0, Rδ1, Rδ2, Rδ3), (G1δ0, G1δ1, G1δ2, G1δ3), (G2δ0, G2δ1, G2δ2, G2δ3), and (Bδ0, Bδ1, Bδ2, Bδ3), wherein the green phase-shifting hologram images (G1δ0, G1δ1, G1δ2, G1δ3) are captured by the unit cell 930, and the green phase-shifting hologram images (G2δ0, G2δ1, G2δ2, G2δ3) are captured by the unit cell 940.
In one embodiment, the green filter 974 and the green filter 976 are respectively formed of a green color filter (for example, “GCF” in
In one embodiment, the green filter 974 and the green filter 976 are respectively formed of a green multi-film (for example, “MFG” in
In one embodiment, the green filter 974 and the green filter 976 are respectively formed of a green grating, the red filter 972 is formed of a red grating, and the blue filter 978 is formed of a blue grating, as shown in
In step S1420, the method involving calculating the object wave in each color channel in the Fourier domain according to the 16 phase-shifting hologram images in different color channels and different phases.
In step S1430, the method involving reconstructing the object image for each color channel according to the object wave in each color channel. Specifically, there are four color channels such as one red channel, one blue channel, and two green channels for the image-sensing device 900 (such as the color image sensor), and the operations for calculating the object wave and reconstructing the object image in a single color channel may be referred to in the embodiment of
Thus, four object images representing one red channel, one blue channel, and two green channels are obtained after step S1430, and an image signal processor (not shown) coupled to the image-sensing device 900 may reconstruct the original color image using the four object images.
In summary, according to the image-sensing device disclosed by the embodiment of the disclosure, the photoelectric elements in each of the unit cells are formed of pillar structures, and the photoelectric elements are different sizes. By arranging photoelectric elements with different sizes, that are designed for different phases in the 4-step phase-shifting holography algorithm, into the sensor array of the image-sensing device, the object image may be reconstructed using the phase-shifting hologram images captured by the photoelectric elements, and thus no modular lens is required in the camera module using the lens-free image sensor, and thus the cost of the whole camera module may be reduced and the thickness of the camera module may be thinner.
While the disclosure has been described by way of example and in terms of the preferred embodiments, it should be understood that the disclosure is not limited to the disclosed embodiments. On the contrary, it is intended to cover various modifications and similar arrangements (as would be apparent to those skilled in the art). Therefore, the scope of the appended claims should be accorded the broadest interpretation to encompass all such modifications and similar arrangements.
Number | Name | Date | Kind |
---|---|---|---|
20130100333 | Awatsuji et al. | Apr 2013 | A1 |
20190041794 | Lin | Feb 2019 | A1 |
Entry |
---|
Hiroyuki Toge et al:“One-shot digital holography for recording color 3-D images”, Proceedings of SPIE, vol. 6912, Feb. 7, 2008 (Feb. 7, 2008), pp. 69120U-69120U-8, XP055113565. |