The present invention relates to a method and a processing unit for adjusting the characteristic curve of the exposure sensitivity of an image sensor, the image sensor being located in particular in a motor vehicle.
Outdoor scenes often have a broad range of different radiation intensities. In particular in traffic scenes in the motor vehicle field, frequently a great difference in radiation intensities occurs. This becomes clear for example when a motor vehicle approaches a tunnel entrance in daylight. While the surroundings of the motor vehicle outside the tunnel are bright in the sunlight, often only weak lighting is found in the tunnel itself. With the plan to equip motor vehicles with image sensors which register the surroundings of the motor vehicle, it is desirable for the image sensors to process this wide range of different radiation intensities. In the conventional regulation of an image sensor, the image sensor is adapted to the lighting conditions through appropriate choice of the parameters of gain, offset, integration time, and/or aperture. In newer developments of image sensors, the exposure sensitivity does not follow a rigidly set linear or logarithmic characteristic curve, but rather the characteristic curve may be adjusted individually in particular in single linear segments. For example, a CMOS image sensor having an adjustable characteristic curve with linear segments is discussed in U.S. Pat. No. 6,348,681, Reference to a method and a processing unit for attaining high information content of generated images is lacking in U.S. Pat. No. 6,348,681.
The method described below for adjusting the characteristic curve of the exposure sensitivity of at least one pixel of at least one image sensor has the advantage that the generated image information has high information content and a broad dynamic range. The method described below is particularly advantageous in outdoor use of the at least one image sensor, since outdoors a plurality of different radiation intensities may occur simultaneously in a scene. In particular, the method described below is suitable for use in a motor vehicle, since with this method, for example, sections of streets which are in full sunlight and simultaneously have areas of shadow may be mapped with high information content of the image information. In an advantageous manner, the method described below utilizes the additional degrees of freedom of a characteristic curve of the exposure sensitivity having segments of functions, in particular continuous and/or linear functions, so that the at least one image sensor captures a maximum of information in its images about the scene being mapped at that time.
It is advantageous to approximate the characteristic curve of the exposure sensitivity to the optimal characteristic curve of the exposure sensitivity through at least one numerical approximation method and/or at least one segmenting method. In particular, approximation through at least one segmenting method has the advantage that the at least one segmenting method contributes to a simple implementation of the method on a microprocessor.
Particularly advantageous is adjustment of the characteristic curve of the exposure sensitivity, the characteristic curve being formed in segments of linear functions. Characteristic curves of the exposure sensitivity having segments of linear functions contribute to the characteristic curves being easily realizable in the image sensor, and also enable a simple algorithmic implementation of the method described below in the processing unit.
It is advantageous to adjust the characteristic curve of the exposure sensitivity of at least one pixel of at least one image sensor as a function of image signals from at least two image sensors, in particular from at least one stereo camera which is made up of at least two sensors. This procedure has the advantage that by averaging the image signals the precision and robustness of the method for adjusting the exposure sensitivity of at least one pixel of at least one image sensor is increased.
The described advantages apply correspondingly to the method described below and/or the processing unit described below and/or the computer program described below.
A computer program having a program code arrangement for carrying out all the steps, or at least the essential steps of the described method, when the program is executed on a computer, is advantageous. The use of a computer program facilitates quick and cost-effective adaptation of the method, for example to different image sensors.
Additional advantages derive from the following description of exemplary embodiments with reference to the figures.
A method and a processing unit for adjusting the characteristic curve of the exposure sensitivity of at least one pixel of at least one image sensor in a motor vehicle are described below. In an exemplary embodiment, the optimal characteristic curve of the exposure sensitivity is determined from the histogram of the gray values of at least one image as a function of image signals from at least one image sensor. The freely definable characteristic curve of the image sensor having linear segments is chosen so that it agrees at least approximately with the optimal characteristic curve. Along with the size and/or the position of the characteristic curve of the exposure sensitivity, alternatively or in addition the shape of the characteristic curve is adjusted.
The image signals generated by image sensor 40 are transmitted over signal line 44 to processing unit 42. The image signals are transmitted electrically and/or optically and/or by radio. From the incoming image signals, i.e., the camera images, the histogram of the gray values is determined in histogram determination module 50. The histogram of the gray values indicates the frequency of the individual gray values of an image or image detail. In the exemplary embodiment, the histogram of the gray values is determined from an overall image. For each gray value, the associated illumination intensity is determined in freely definable, virtual units in histogram determination module 50 as a function of the parameters set, such as the gain and/or the offset and/or the integration time, and/or the parameters for adjusting the characteristic curve of the exposure sensitivity, such as the number of segments of the characteristic curve of the exposure sensitivity, and/or the position of the segments of the characteristic curve of the exposure sensitivity, and/or the size of the segments of the characteristic curve of the exposure sensitivity, and/or the at least one parameter of the linear functions.
In module 52 for determining the optimal characteristic curve, the optimal characteristic curve is calculated as a function of the histogram of the gray values determined in preceding histogram determination module 50. With N gray values in the histogram, having respective frequencies hj and associated illumination intensities Φj, the result is a table of N elements, whose entries i are calculated for example from
where Gmax designates the maximum depictable gray value. The calculated function for mapping illumination intensities Φ on gray values g represents the optimal characteristic curve of the exposure sensitivity according to information theory for the given histogram, and is an optimal characteristic curve of the exposure sensitivity.
The optimal characteristic curve of the exposure sensitivity is distinguished basically by the fact that after adjustment of the characteristic curve of the exposure sensitivity of the image sensor, which is approximated to the determined optimal characteristic curve of the exposure sensitivity, the frequency of the individual gray values in the histogram of the image signals, i.e., of the camera image of the image sensor, is approximately constant, and/or the gray value density of the histogram of the image signals, i.e., of the camera image of the image sensor, is approximately constant. The frequency of a gray value here designates the number of pixels within the camera image that have this gray value based on the total number of pixels. A constant frequency of the gray values within a histogram of an image is referred to as a uniformly distributed histogram. The gray value density designates the sum of frequencies h(gi) of gray values gi in an interval Δg of gray values in reference to this interval Δg: (Σh(gi))/Δg.
In module 54 for approximating the characteristic curve of the exposure sensitivity, the characteristic curve of the exposure sensitivity in linear segments of the pixels of the image sensor is now chosen so that it agrees as precisely as possible with the optimal characteristic curve obtained. In the exemplary embodiment, this is performed in module 54 for approximating the characteristic curve of the exposure sensitivity, through a numerical minimization of the quadratic separation between the optimal characteristic curve of the exposure sensitivity and the characteristic curve in linear segments, while varying the parameters.
Alternatively or in addition, in a variant of the exemplary embodiment, the segmenting method described in Hwa-Cho Yi: Sensordatenauswertung mit Fuzzy-Logik fuer das automatisierte Entgraten [Sensor data evaluation with fuzzy logic for automated deburring] (Munich: Hanser-Verlag, 1993, TU-Berlin dissertation, pp. 76-79) is used. This segmenting method operates according to the iterative final point method or the principle of the search for the longest segment. The determined parameters, such as the gain and/or the offset and/or the integration time and/or number of segments of the characteristic curve of the exposure sensitivity and/or the position of the segments of the characteristic curve of the exposure sensitivity and/or the size of the segments of the characteristic curve of the exposure sensitivity, and/or at least one parameter of the linear functions, are transmitted from module 54 for approximating the characteristic curve of the exposure sensitivity to module 56 for generating the adjustment signals.
In module 56 for generating the adjustment signals, the at least one corresponding adjustment signal is determined from the determined parameters, and the at least one adjustment signal is transmitted over signal line 46 electrically and/or optically and/or by radio to image sensor 40. In image sensor 40, the characteristic curve of the exposure sensitivity is adjusted as a function of at least one adjustment signal. In the exemplary embodiment, the characteristic curve of the exposure sensitivity takes effect in the image that follows determination of the optimal characteristic curve of the exposure sensitivity. Alternatively or in addition, in additional variants the characteristic curve of the exposure sensitivity takes effect in the same image and/or in the next image but one and/or in at least one additional image.
The use of the method according to the present invention and/or of the processing unit according to the present invention and/or of the computer program according to the present invention for adjusting the characteristic curve of the exposure sensitivity of at least one pixel of at least one image sensor becomes clear when the at least one image sensor receives an incoming image having a wedge of gray values, the wedge of gray values being made up of two segments having a different gradient of the gray values. In otherwise adjusted and/or controlled and/or regulated image sensors having a characteristic curve in linear segments, with variable overall illumination of the gray value wedge, an apparent contour caused by the bent characteristic curve of the exposure sensitivity appears, which does not agree with the segment boundary of the gray value wedge. In contrast, through the method according to the present invention and/or the processing unit according to the present invention and/or the computer program according to the present invention, this apparent contour will lie exactly on or at least very close to the separation line, even if the overall illumination is changed.
The described method and/or processing unit and/or computer program are not restricted to adjusting the characteristic curve of the exposure sensitivity of an image sensor in a motor vehicle. Rather, the described approach using the corresponding features is also usable outside of motor vehicle technology, moreover with more than one image sensor. Furthermore, the described method and/or processing unit and/or computer program is not restricted to adjusting the characteristic curve of the exposure sensitivity. Rather, the approach is also suitable in particular for controlling and/or regulating the characteristic curve of the exposure sensitivity.
In a variant of the described method and/or the processing unit and/or the computer program, the characteristic curve of the exposure sensitivity of at least one pixel of at least one image sensor is adjusted as a function of at least one image detail from at least one image sensor. A first embodiment includes the provision that the image segment is specifiable manually by at least one user, while in an additional embodiment the image detail is determined automatically by the processing unit.
Another variant contains the provision, alternatively or in addition, that the characteristic curve of the exposure sensitivity of at least one pixel of at least one image sensor is adjusted as a function of image signals, in particular of at least one image and/or at least one image detail, from at least two or more image sensors. Alternatively or in addition, the adjustment of the characteristic curve of the exposure sensitivity of at least one pixel of at least one image sensor is performed as a function of image signals from at least one stereo camera. The stereo camera is made up here of at least two image sensors, which record essentially the same scene. For example, a common histogram of the gray values is calculated here from the images and/or image details and is used to determine the optimal characteristic curve of the exposure sensitivity.
In general, the described procedure is not restricted to a black/white CMOS image sensor having a resolution of 640×480 pixels with 8-bit gray scale resolution. Rather, the method and/or the processing unit and/or the computer program are suitable for all types of image sensors. The only prerequisite is that the image sensors have a configurable characteristic curve of the exposure sensitivity, having segments with functions, in particular continuous and/or linear functions. For example, in one variant a CCD image sensor is used. In another variant, alternatively or in addition, an image sensor having a logarithmic characteristic curve in segments is used. In addition, the described approach using the corresponding features is also usable with at least one color image sensor. Alternatively or in addition, in another variant the characteristic curve of the exposure sensitivity of at least one pixel of at least one image sensor is adjusted.
Number | Date | Country | Kind |
---|---|---|---|
10301898.0 | Jan 2003 | DE | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/DE03/03744 | 11/12/2003 | WO | 4/12/2006 |