Patent Application
20110164146
1. Field of the Invention
The present invention generally relates to an image sensor, and more particularly to adaptively controlling the frame rate in an image sensor.
2. Description of Related Art
Semiconductor based image sensors such as complementary metal-oxide-semiconductor (CMOS) sensors are widely used in, for example, cameras and camcorders, to capture light energy and convert it to electrical signals. The signals are amplified, digitized and then processed to construct an image. The amount of the light energy captured by the image sensor is proportional to the exposure time during which the active sensing area of the image sensor is exposed. Thus, longer exposure times result in more energy being collected.
The length of the exposure time is generally determined by two primary factors: the measured incident light relative to the desired signal level, and the sensor frame rate. Regarding the former, in a dark lighting condition, for example, longer exposure time is required in order to reach the desired signal level. Regarding the latter, the frame rate is mostly determined according to the relevant application, and it sets the limits of the maximum exposure time.
Amplifiers are conventionally used to amplify the sensor output signals to achieve the desired signal level. Compared to increasing exposure time, amplifying the sensor output signal will never decrease the frame rate. However, since increasing signal gain disadvantageously increases noise and therefore reduces overall image quality, such is therefore typically used only when more signal is required beyond the maximum exposure limit.
For the reason that conventional image sensing systems or methods could not effectively optimize sensor noise performance, a need has arisen to propose a novel system and method for dynamically controlling the frame rate in the image sensor in order to optimize the sensor noise performance (e.g., signal-to-noise ratio, SNR) with respect to the frame rate.
In view of the foregoing, it is an object of the embodiment of the present invention to provide a system and method for adaptively controlling the frame rate in an image sensor in order to optimize sensor noise performance (e.g., signal-to-noise ratio, SNR).
According to the embodiments of the present invention, an amplifier, such as a programmable gain amplifier, amplifies a sensor output signal of the image sensor with a gain. An automatic exposure gain (AEG) controller determines a product of exposure time and the gain (EGP) based on incident light intensity, and controls a timing controller and the programmable gain amplifier, thereby adjusting the exposure time and the gain of the image sensor such that the amplified sensor output signal may approximately approach a sensor target signal, wherein the change of the exposure time is preferred over the change of the gain. In one embodiment, multiple sets of maximum exposure limits and gain limits are provided by the AEG controller in respective time intervals to the timing controller and the amplifier, respectively, in adjusting the exposure time and the gain of the image sensor. In another embodiment, multiple sets of sensor clocks and gain limits are provided by the AEG controller in respective time intervals to the timing controller and the amplifier, respectively, in adjusting the exposure time and the gain of the image sensor.
Specifically, the AEG controller 16 firstly determines the value of the product of the exposure time and the gain (“exposure gain product” or EGP) of a current frame or frames required to reach the desired sensor target signal based on the intensity of incident light. Based on the determined EGP, one or more of the exposure time and the gain are then adaptively adjusted to optimize sensor noise performance (e.g., signal-to-noise ratio, SNR) with respect to frame rate. In the embodiment, increasing the exposure time is preferred over increasing the gain for the reason that more signal amplification (or gain) introduces more noise and therefore reduces overall image quality. Likewise, reducing the gain is preferred over reducing the exposure time. For instance, an algorithm may favor changes in exposure time over non-reducing changes in (i.e., adjustments that do not decrease, that change, and/or that increase) the gain, or an algorithm may exhibit a preference in changing (or a bias or propensity to change) exposure time over increasing the gain. This type of approach, preference, paradigm, or algorithm (e.g., rule) is generally adopted in the embodiment under some constraints that will become evident from the following paragraphs.
Referring to
In an operation of the adaptive frame rate control system 1, as the incident light intensity, for example, decreases in the first time interval t1-t2, the gain provided by the AEG controller 16 (to the amplifier 12) accordingly increases toward but not over/exceeding the maximum gain limit g2, while the exposure time provided by the AEG controller 16 (to the timing controller 18) is maintained at the maximum exposure limit 1 (e1). It is noted that a first frame rate r1 is correspondingly obtained in this time interval t1-t2.
When the incident light intensity further decreases to the extent that the determined exposure gain product (EGP) exceeds the product of the maximum exposure limit 1 (e1) and the maximum gain limit g2, that is, EGP>e1×g2, another maximum exposure limit (e.g., the maximum exposure limit 2 (e2)) now replaces the preceding one. As the incident light intensity, for example, decreases in the second time interval t2-t3, the gain provided by the AEG controller 16 (to the amplifier 12) accordingly increases from the minimum gain limit g1 toward but not over/exceeding the maximum gain limit g2, while the exposure time provided by the AEG controller 16 (to the timing controller 18) is maintained at the maximum exposure limit 2 (e2). It is noted that a (lower) second frame rate r2 is correspondingly obtained in this time interval t2-t3.
Likewise, when the incident light intensity further decreases to the extent that the determined exposure gain product (EGP) exceeds the product of the maximum exposure limit 2 (e2) and the maximum gain limit g2, that is, EGP>e2×g2, another maximum exposure limit (e.g., the maximum exposure limit 3 (e3)) now replaces the preceding one. As the incident light intensity, for example, decreases in the third time interval t3-t4, the gain provided by the AEG controller 16 (to the amplifier 12) accordingly increases from the minimum gain limit g1 toward but not over/exceeding the maximum gain limit g2, while the exposure time provided by the AEG controller 16 (to the timing controller 18) is maintained at the maximum exposure limit 3 (e3). It is noted that a (even lower) third frame rate r3 is correspondingly obtained in this time interval t3-t4. It is assumed that, in the embodiment, the third frame rate r3 is the acceptably lowest frame rate, below which motion blur incurs. Consequently, the sensor output signal deviates from the sensor target signal while the incident light intensity still decreases and the exposure time and the gain are maintained unchanged as shown in
According to the operation described above, once the exposure limit has been set, the gain is applied from the minimum gain limit g1 until the maximum gain limit g2 is reached. The AEG controller 16 accordingly redistributes the exposure gain product in favor of applying higher exposure than gain. The redistribution will result in reduction of the frame rate and the gain, thereby improving noise performance and obtaining better signal-to-noise ratio (SNR).
As the preceding paragraphs describe the operation concerning the decreased incident light intensity, the timing diagrams may be traced backward for describing the conditions in which the incident light intensity is increasing from low to high. In other words, the maximum exposure limit may be changed from the maximum exposure limit 3 (e3) to the maximum exposure limit 2 (e2) (in the time interval t2-t3) when the incident light intensity increases to the extent that the determined exposure gain product (EGP) becomes smaller than the product of the maximum exposure limit 3 (e3) and the minimum gain limit g1, that is, EGP<e3×g1. The maximum exposure limit may be further changed from the maximum exposure limit 2 (e2) to the maximum exposure limit 1 (e1) (in the time interval t1-t2) when the incident light intensity still increases to the extent that the determined exposure gain product (EGP) becomes smaller than the product of the maximum exposure limit 2 (e2) and the minimum gain limit g1, that is, EGP<e2×g1.
Conclusively speaking, a distinct exposure limit is set whenever the maximum gain limit g2 or the minimum gain limit g1 has been reached in the respective time interval. Specifically, the exposure limit is decreased in a case where the maximum gain limit g2 has been reached in the respective time interval, and the exposure limit is increased in a case where the minimum gain limit g1 has been reached in the respective time interval.
In the present embodiment, multiple sensor clocks, rather than the exposure limits of the previous embodiment, are provided (from the AEG controller 16) to the timing controller 18. As the sensor clock decides the exposure time, each sensor clock determines its respective exposure time. For example, the sensor clock 1 corresponds to a first exposure time e1, the (smaller) sensor clock 2 corresponds to a second exposure time e2, and the (even smaller) sensor clock 3 corresponds to a third exposure time e3.
The operation (in
Although specific embodiments have been illustrated and described, it will be appreciated by those skilled in the art that various modifications may be made without departing from the scope of the present invention, which is intended to be limited solely by the appended claims.
