The present invention relates to an image sensing method and an image sensor with rolling exposure time compensation; particularly, it relates to such image sensing method and such image sensor with rolling exposure time compensation which adaptively determine an exposure time of at least one sensor row according to a brightness signal of an image.
There are two types of shutters for rolling exposure in conventional image sensors: mechanical shutter and electronics shutter. However, regardless whether a mechanical shutter or electronics shutter is used, due to certain reasons such as the location of the light source, there is a problem of uneven brightness between different rows or columns in an image, causing it hard to correctly identify the image which is for example a fingerprint.
In view of the above, to overcome the drawback in the prior art, the present invention proposes an image sensing method and an image sensor with rolling exposure time compensation, which adaptively determine an exposure time of at least one sensor row according to a brightness signal of an image.
From one perspective, the present invention provides an image sensing method with rolling exposure time compensation, comprising: emitting light to an object to generate an image; providing a sensor device having a plurality of sensor units which are arranged by a plurality of columns and a plurality of rows; sensing the image by the sensor device by way of rolling exposure; and adaptively determining an exposure time of at least one row of the sensor device according to an image brightness signal and controlling a rolling shutter according to the exposure time when the sensor device senses the image.
In one preferable embodiment, the step of adaptively determining an exposure time of at least one row of the sensor device according to an image brightness signal includes: in a preparation period, controlling the rolling shutter to sense the image by the sensor device by way of rolling exposure, wherein the at least one row is exposed for a first exposure period which has a predetermined time length, to generate a preparation scan image, and the image brightness signal is generated thereby; and in a compensation period after the preparation period, determining a second exposure period of the at least one row according to the image brightness signal; wherein the exposure time includes the second exposure period.
In the aforementioned embodiment, the step of determining a second exposure period of the at least one row according to the image brightness signal preferably further includes: determining the second exposure period in the compensation period according to a predetermined brightness target.
In the aforementioned embodiment, the image brightness signal is preferably obtained by smoothing an original image brightness signal.
In one preferable embodiment, a light source for emitting the light to the object is located outside and neighboring a first row and/or a last row of the plural rows.
From another perspective, the present invention provides an image sensing method with rolling exposure time compensation, comprising: emitting light to an object to generate an image; providing a sensor device having a plurality of sensor units which are arranged by a plurality of columns and rows; sensing the image by the sensor device by way of rolling exposure; in a preparation period, controlling the rolling shutter to sense the image by the sensor device by way of rolling exposure, wherein at least one row of the sensor device is exposed for a first exposure period which has a predetermined time length, to generate a preparation scan image, and an image brightness signal is generated thereby; and in a compensation period after the preparation period, determining at least one unit exposure time of the at least one row according to the image brightness signal to generate at least one unit scan image, and generating combination image information according to the preparation scan image and the at least one unit scan image.
In one preferable embodiment, the step of determining at least one unit exposure time of the at least one row according to the image brightness signal to generate at least one unit scan image, and generating combination image information according to the preparation scan image and the at least one unit scan image, includes: generating a brightness contour according to the image brightness signal and a bright threshold; and determining the at least one unit exposure time of the at least one row according to the brightness contour, to generate the at least one unit scan image
In one preferable embodiment, the step of determining at least one unit exposure time of the at least one row according to the image brightness signal to generate at least one unit scan image, and generating combination image information according to the preparation scan image and the at least one unit scan image, includes: determining the at least one unit exposure time of the at least one row according to a single row brightness signal of the image brightness signal, to generate the at least one unit scan image.
In one preferable embodiment, a light source for emitting the light to the object is located outside and neighboring a first row and/or a last row of the plural rows.
In one preferable embodiment, the object includes a finger, and the image includes a fingerprint image, and the light source emitting the light to the finger to generate the fingerprint image.
From another perspective, the present invention provides an image sensor with rolling exposure time compensation, for sensing an image, comprising: a sensor device, which includes a plurality of sensor units arranged by a plurality of columns and rows, and is configured to operably sense the image by way of rolling exposure; a light source, which is configured to operably emit light to an object for generating the image; and an exposure time compensation circuit, which is coupled to the sensor device, and is configured to adaptively determine an exposure time of at least one row of the sensor device according to an image brightness signal and control a rolling shutter accordingly when the sensor device senses the image by way of rolling exposure.
In one preferable embodiment, the exposure time compensation circuit controls the rolling shutter such that when the sensor device senses the image by way of rolling exposure in a preparation period, the at least one row is exposed for a first exposure period which has a predetermined time length, whereby the sensor device generates a preparation scan image, and the image brightness signal is generated thereby; and in a compensation period after the preparation period, the exposure time compensation circuit determines a second exposure period of the at least one row according to the image brightness signal, wherein the exposure time includes the second exposure period.
In the aforementioned embodiment, the exposure time compensation circuit preferably determines the second exposure period in the compensation period according to a predetermined brightness target.
In the aforementioned embodiment, the image brightness signal is preferably obtained by smoothing an original image brightness signal.
In one preferable embodiment, the light source for emitting the light to the object is located outside and neighboring a first row and/or a last row of the plural rows.
From another perspective, the present invention provides an image sensor with rolling exposure time compensation, for sensing an image, comprising: a sensor device, which includes a plurality of sensor units arranged by a plurality of columns and rows, and is configured to operably sense the image by way of rolling exposure; a light source, which is configured to operably emit light to an object for generating the image; and an exposure time compensation circuit, which is coupled to the sensor device, and is configured to operably control a rolling shutter, such that when the sensor device senses the image by way of rolling exposure in a preparation period, at least one row of the sensor device is exposed for a first exposure period which has a predetermined time length, whereby the sensor device generates a preparation scan image, and an image brightness signal is generated thereby; and in a compensation period after the preparation period, the exposure time compensation circuit determines at least one unit exposure time of the at least one row according to the image brightness signal, such that the sensor device generates at least one unit scan image, and combination image information is generated according to the preparation scan image and the at least one unit scan image.
In one preferable embodiment, the exposure time compensation circuit generates a brightness contour according to the image brightness signal and a bright threshold, and determines the at least one unit exposure time of the at least one row according to the brightness contour, to generate the at least one unit scan image.
In the aforementioned embodiment, the exposure time compensation circuit preferably determines the at least one unit exposure time of the at least one row according to a single row brightness signal of the image brightness signal, to generate the at least one unit scan image.
In the aforementioned embodiment, the light source for emitting the light to the object is preferably located outside and neighboring a first row and/or a last row of the plural rows.
In one preferable embodiment, the object includes a finger, and the image includes a fingerprint image, and the light source emitting the light to the finger to generate the fingerprint image.
The objectives, technical details, features, and effects of the present invention will be better understood with regard to the detailed description of the embodiments below, with reference to the attached drawings.
The drawings as referred to throughout the description of the present invention are for illustration only, to show the interrelations between the circuits and the signal waveforms, but not drawn according to actual scale.
Please refer to
“Rolling exposure” is a method of exposing an image frame row by row, wherein a next row starts scanning after a previous row starts scanning but before this previous row finishes scanning, which is well known to those skilled in this art and therefore is not redundantly explained here. Rolling exposure can be carried out by an electronic shutter or a mechanical shutter. The electronic shutter is used commonly in a complementary metal oxide semiconductor (CMOS) image sensor, wherein the exposure scanning operation is controlled by electronic signals to scan the sensor units in the image sensor row by row from top to bottom (or from bottom to top).
According to the present invention, a light source is provided to emit light to the object for generating the image. For example, the object may be a finger, and the light source is for example but not limited to alight emitting diode (LED) device. When the LED device emits light to the finger, the light scatters, refracts, and reflects according to the fingerprint pattern, whereby a fingerprint image is generated. The method of generating the fingerprint image is known as “light finger”. The sensor device for example may be, but is not limited to, a CMOS image sensor, which has plural CMOS image sensor units arranged by plural columns and plural rows.
As explained, the rolling shutter exposes the CMOS image sensor units of the CMOS image sensor row by row, to receive for example the fingerprint image. However, “row-by-row” does not strictly mean that only one single row is exposed at one time; plural rows can be exposed at the same time. And, for one image frame such as a fingerprint image, it is not limited for the sensor device to scan the fingerprint image only once; the sensor device can scan the fingerprint image multiple times.
According to the present invention, the rolling shutter is controlled to adaptively determine the exposure time of at least one row of the sensor device according to an image brightness signal when the sensor device senses the image by way of rolling exposure. That is, the exposure time of different rows may be different according to the image brightness signal. The image brightness signal will be described in detail later.
The present invention is different from the prior art in at least one aspect that, according to the present invention, the exposure time of different rows maybe adaptively adjusted according to the image brightness signal, such that the overall brightness of an image picture generated according to the present invention can be relatively even between different rows, and the image can be displayed clearer.
As shown in
In the embodiment show in
In one embodiment, the exposure time compensation circuit controls the rolling shutter, such that when the sensor device senses the image by way of rolling exposure in a preparation period, the at least one row is exposed for a first exposure period which has a predetermined time length, whereby the sensor device generates a preparation scan image, and the image brightness signal is generated according to the preparation scan image; and in a compensation period after the preparation period, the exposure time compensation circuit determines a second exposure period of the at least one row according to the image brightness signal, wherein the exposure time includes the second exposure period.
As shown in
More specifically, in this embodiment, the light sources are located outside and neighboring the last column of the plural columns, and the rolling shutter exposes the rows sequentially (row-by-row or rows-by-rows), so the scanning direction and the brightness distribution (increasing along the lateral direction) are different. Therefore, the rolling exposure time compensation method according to the first embodiment, i.e., adaptively adjusting exposure time of the rows, still needs to face an issue that the brightness is not uniform along the lateral direction. In this second embodiment, the image brightness signal is compared with a bright threshold, to generate a brightness contour. For example, the brightness contour may be formed by a group of contour points, and the points are obtained by the following steps: first, comparing the brightness of each sensor unit of each row with a brightness threshold along the lateral direction; second, in each row, determining a last sensor unit whose brightness is not higher (or higher, or a first sensor unit whose brightness is not lower) than the brightness threshold as a contour point of the group; and third, forming the contour by the contour points.
Next, for example, the highest brightness among the sensor units of the image brightness signal is selected as a brightness target in the preparation period, for example but not limited to a brightness of a sensor unit of the last column which is located nearest to the light source. Next, in this embodiment, the unit exposure time of each row is individually determined by for example the aforementioned heterodyne method, according to the brightness target and a contour point in that row, wherein different rows may have different unit exposure times. Next, in the compensation period, the unit scan image is generated with rolling exposure time compensation, wherein the exposure time of each row is the individually calculated unit exposure time. Next, at least one unit scan image is generated (in one embodiment, plural unit scan images are generated by repeating the aforementioned steps of generating the brightness contour and the unit scan image, until the brightness of every the sensor unit achieves the brightness target in at least one unit scan image). Next, by high dynamic range imaging (HDR) method, combination image information is generated by combining the preparation scan image and the at least one unit scan image. The HDR method is well known to those skilled in this art and therefore is not redundantly explained here.
As shown in
The exposure time compensation circuit determines at least one unit exposure time of the at least one row according to the image brightness signal, whereby the sensor device generates at least one unit scan image, and the combination image information is generated according to the preparation scan image and the at least one unit scan image.
Next, for example, the highest brightness in the single row is selected as a brightness target in the preparation period, for example but not limited to a brightness of a sensor unit of the last column of the signal row which is located nearest to the light source. Next, the unit exposure time of each row is determined by for example the aforementioned heterodyne method, according to the brightness target and the boundary brightness. Next, at least one of the unit scan image is generated (in one embodiment, plural unit scan images are generated by repeating the aforementioned steps of generating the brightness contour and the unit scan image, until the brightness of every the sensor unit achieves the brightness target in at least one unit scan image). Next, by high dynamic range imaging (HDR) method, combination image information is generated by combining the preparation scan image and the at least one unit scan image.
The present invention has been described in considerable detail with reference to certain preferred embodiments thereof. It should be understood that the description is for illustrative purpose, not for limiting the scope of the present invention. An embodiment or a claim of the present invention does not need to achieve all the objectives or advantages of the present invention. The title and abstract are provided for assisting searches but not for limiting the scope of the present invention. Those skilled in this art can readily conceive variations and modifications within the spirit of the present invention. It is not limited for each of the embodiments described hereinbefore to be used alone; under the spirit of the present invention, two or more of the embodiments described hereinbefore can be used in combination. For example, two or more of the embodiments can be used together, or, a part of one embodiment can be used to replace a corresponding part of another embodiment. In view of the foregoing, the spirit of the present invention should cover both such and other modifications and variations, which should be interpreted to fall within the scope of the following claims and their equivalents.
Number | Date | Country | Kind |
---|---|---|---|
106129510 | Aug 2017 | TW | national |
The present invention claims priority to U.S. Ser. No. 62/440746, filed on Dec. 30, 2016 and claims priority to TW 106129510 filed on Aug. 30, 2017.
Number | Date | Country | |
---|---|---|---|
62440746 | Dec 2016 | US |