The invention relates to a device and a method for detecting raindrops on a pane by means of a lighting source and a camera.
In WO2010/072198 A1 rain detection is described with the aid of a camera, which is used for automotive driver assistance functions. For rain detection bifocal optics are used, which form a sharp image of a portion of the windscreen onto a portion of the image chip or image sensor of the camera.
A disadvantage of this idea is the fact that an additional optical element is introduced, whose edge causes serious disturbances both in the beam path for the rain sensor area of the image chip as well as in the region for the driver assistance functions in the vicinity of the edge. In particular for implementations with small dimensions, the focus conditions for the driver assistance and the rain sensor area are strongly different, what must be compensated by an increased thickness of the optical element, thus leading to increased disturbances and a broad, non-usable region on the image chip around the edge.
Another disadvantage results from different pane inclinations, which have different optical distances between rain sensor detection area on the image chip and corresponding rain sensor surface on the pane. To continue to ensure a sharp optical imaging, for each modified installation situation the thickness of the optical element must be adapted.
In order to detect raindrops also at night, it is proposed in WO 2010/072198 A1 to couple light via a coupling element into the windscreen and to guide it via total reflection in the pane. By a decoupling element the totally reflected light is decoupled in the direction of the camera. When there are water drops on the windscreen, a part of the light is decoupled and is no longer totally reflected to the decoupling element. It is also disadvantageous here that for each modified pane inclination the integrated camera lighting unit must be mechanically adapted to the modified installation condition.
In U.S. Pat. No. 7,259,367 B2 also by means of a camera rain sensing is proposed, which provides a large-area lighting of the passing-through window of the camera opening angle with the pane. The camera focus is set to almost infinite and thus can be simultaneously used for driver assistance applications. Because of the imaging on the far range raindrops are noticeable only as disturbances in the image, which are detected by complex differential measurements of the images recorded with light pulsed or modulated in synchronization with the pixel clock.
However, computer simulations and measurements show that with this type of lighting only a very small portion of the light is reflected at the raindrops back into the camera. This fact leads to a poor signal to noise ratio and consequently to an insecure rain detection.
It is, therefore, an object of an embodiment of the present invention to overcome the mentioned disadvantages of the devices or methods known from prior art.
This object can be achieved by a device for detecting rain, according to an embodiment of the invention, which comprises a camera and a lighting source. The camera is disposed behind a pane, in particular in the interior of a vehicle e.g. behind a windscreen, and is focused onto a far range that lies in front of the pane. The camera preferably comprises an objective for focusing and an image sensor, e.g. a CCD or CMOS sensor. The lighting source for generating at least one light beam directed towards the pane directs the at least one light beam towards the pane, such that at least one beam that is reflected from the outer face of the pane (or partial beam of the light beam directed towards the pane) impinges on the camera. The lighting source can be embodied as one or more light emitting diodes (LEDs) or as a light band.
The light quantity of the beam impinging on the at least one camera can be measured by the camera.
The invention provides a simple but reliable possibility to detect rain with a vehicle camera, in particular with a driver assistance camera. Since essentially only one light quantity is to be measured, no complex image processing algorithm is needed. By the active lighting the device is relatively less interference-prone to external influences such as sun reflexes and hard shadows.
The rain quantity, for example with multi-beam lighting sources, can be determined via a reduction of the light quantity of all light reflexes (on the image sensor) of the outer windscreen and/or via the number of the influenced light reflexes.
In a preferred form of embodiment the angle of incidence of the light beam generated by the lighting source is set such that the beam (part), which impinges on the outer face of the pane, is more reflected than decoupled from the pane, when there is no rain on the outer face of the pane.
According to an advantageous form of embodiment, the device comprises an evaluation unit, which determines from the measured light quantity of the beam that is reflected at the outer face of the pane whether and if so how much rain is present on the outer face of the pane.
Preferably, for determining rain the evaluation unit can compare the measured light quantity of the beam that is reflected at the outer face of the pane with a threshold value. The threshold value may be adapted in particular to a modified intensity of the lighting and/or to a modified sensitivity of the camera, e.g. by regular calibrations with a dry pane. Also, several threshold values can be used.
Advantageously, the evaluation unit determines a time variation of the light values measured by the image sensor of the camera of the beam that is reflected at the outer face of the pane. For this purpose, a series of images can be recorded with the camera.
In a preferred form of embodiment, the lighting source directs the at least one light beam towards the pane, such that the beams reflected from the inner and outer face of the pane impinge as at least two spatially separated beams on the camera. The light quantities of the at least two beams impinging on the camera can be measured in this case by the camera. The beam, reflected (directly) on the inner face of the pane, which impinges on the camera, in this case preferably serves as a reference signal, since the light quantity of this beam remains constant in case of presence or absence of raindrops on the outer face of the pane.
This type of detection with the lighting proposed here is not necessarily dependent on a camera, but can be utilized by means of each optical sensor, which can determine the light quantities of two spatially separated beams. The advantage of the introduced method of detection compared to conventional diode-rain sensors is based on the fact that no coupling optics is needed and simultaneously, a reference beam is available for a comparative measurement.
Preferably, the camera is used for one or more further driver assistance functions, which are based on an evaluation of the far range imaged in focused manner.
According to an advantageous embodiment of the invention, the lighting source is structurally integrated into the camera or into the housing of the camera. Here, the lighting source can be preferably disposed within the camera housing below a view shield or a view funnel of the camera.
Advantageously, here the lighting source generates light in the infrared wavelength range, and the view shield is permeable in the infrared wavelength range at least in a portion, which is located above the lighting source or in the beam direction of the lighting source.
In this case, the lighting source can be arranged in particular on a circuit carrier or a circuit board of the camera.
Preferably, the lighting source generates only light having a wavelength in a certain wavelength range, such as e.g. in the (near) infrared wavelength range.
In the beam path of the camera, a first spectral filter is arranged in that region, in which the at least two spatially separated reflected beams are passing through. The first spectral filter is at least to a large extent permeable to light having a wavelength in this specific wavelength range (e.g. infrared-permeable).
Advantageously, a second spectral filter is arranged in that region of the beam path, in which the at least two spatially separated reflected beams do not pass through, wherein the second spectral filter blocks light having a wavelength in the specific wavelength range (e.g. IR-cut filter).
The first or both spectral filters can preferably be applied directly on pixels of the image sensor of the camera.
In an advantageous form of embodiment, the lighting source generates a focused light beam.
Preferably, the light beam generated by the lighting source can be directed onto the pane by means of a light guide such as e.g. an optical fiber.
The invention further relates to a method for detecting rain on the outer face of a pane. Also for this, the prerequisites are a camera disposed behind the pane, which is focussed on a far range in front of the pane, and a lighting source for generating at least one light beam directed towards the pane. The lighting source directs the at least one light beam towards the pane such that at least one beam that is reflected from the outer face of the pane impinges on the camera. The light quantity of the at least one beam that is reflected from the outer face of the pane is measured by means of the camera. By evaluating the measured light quantity of the at least one beam that is reflected from the outer face of the pane, the presence or absence of rain on the outer face of the pane can be determined.
A preferred method for detecting rain on the outer face of a pane makes use of a device according to an embodiment of the invention. With the camera at first, a first image is recorded with the lighting source turned off. Then a second image is recorded with the lighting source turned on. The difference image from the second image and the first image is formed. In the difference image, the light quantity of the at least one beam that is reflected at the outer face of the pane is evaluated for detecting rain on the outer face of the pane.
In an advantageous use of visible light as lighting it must be ensured that road users are not disturbed by the lighting.
For this purpose, it is proposed to use a short visible light pulse, adapted by the intensity of the external brightness. This would require only a short exposure time and image recording time for the rain sensor image, what in turn has little influence on the driver assistance function. Such a light pulse would be seen during daylight only when looking directly at the lighting. At night only little light is required for rain detection. Here, the intensity can be down-regulated appropriately, so that also at night the lighting does not have a disturbing effect.
A preferred adaptation of the lighting intensity—regardless of the used wavelength range—provides a further advantage. The rain sensor light reflexes are also clearly visible during the day and at night it is avoided that the images are in saturation and would thus prevent a quantitative evaluation.
The lighting can be advantageously realized via individual LEDs, which e.g. are arranged in series. Alternatively, a light band could be used. Preferably, here a sufficiently directed radiation characteristics of e.g. less than ±20° is guaranteed.
In the following, the invention will be explained on the basis of figures and exemplary embodiments.
A light beam (h) generated by a lighting source (3) is directed towards the pane (2) such that the beams reflected from the inner (2.1) and outer face (2.2) of the pane impinge as two spatially separated beams (rl, r2) on the objective or the camera (1). Due to the focussing on the far range, the boundary of the beam bundle is imaged only blurred on the image chip (5). But both beams (r1, r2) are sufficiently separated and their respective light quantity can be measured with the image sensor (5).
In this form of embodiment, the main beam (h) of the lighting source (3) is used, therefore, the light of the lighting source can be preferably focused. The portion (r1) reflected at the air-pane-interface (or pane inner face (2.1)) of the main beam serves as a reference beam. From the portion which is transmitted into the pane (t1), that portion is used as a measurement beam (r2), which is reflected at the pane-air-interface (or pane outer face (2.2)) and impinges on the camera (1). Not shown is that portion of the beam, which is repeatedly reflected inside the pane (2) (on the inner face (2.1) pane-air, after it was reflected at the outer face (2.2) pane-air).
This arrangement offers the advantage of a distinct signal change with rain (4) on the pane outer face (2.2), as is explained with reference to
When the outer face (2.2) of the windscreen (2) is wetted with rain (4) the majority of light (tl) is decoupled as a decoupled transmitted beam (t2′), so that the reflected portion (r2′) is weakened accordingly (see
By comparing the measured light quantities of both beams (r1 to r2 or r2′), thus the signal (r2′) reduced in case of rain (4) can be easily measured and a windscreen wiper can be activated accordingly.
In order not to irritate the driver and other road users by the lighting (3), in particular near infrared light can be used, for which usually the used CCD or CMOS imaging chips (5) have a high sensitivity.
To become insensitive to disturbances such as noise, daylight and sunlight and other sources of artificial light, it is proposed to partially or completely timely modulate the lighting source (3) preferably synchronously with the image readout clock, so that disturbances can be deducted via simple differential methods. This is one way to improve the signal to noise ratio. A further possibility consists in an appropriate spectral filtering: the section of the image chip (5), on which the beam pairs (r1, r2/r2′) impinge, can be provided with a spectral band pass, which has a high permeability for the wavelength of lighting (3).
In order to realize driver assistance functions simultaneously with the camera image, the light beam pairs (8, 9) may not interfere with the driver assistance image (7). For this purpose, a portion is selected (6) in
In order to avoid disturbances by the lighting (3) as far as possible, in addition an infrared cut filter can be vapor-deposited on a cover glass of the image chip (5) up to the upper edge of the driver assistance area (7). In addition, as already mentioned above, a band-pass filter for the wavelength of lighting (3) can be vapor-deposited above the rain sensor detection area (6).
Alternatively, the filter may also be applied directly to the pixels of the image sensor (5). This would have the advantage that a parallax offset is avoided, which is generated by the edge of the different filters for the rain sensor area (6) and the driver assistance area (7) on the cover glass. A process would be advantageous here, which corresponds to the current application of the pixel color filter. Thereby, the two areas (6, 7) can be separated pixel accurately, avoiding additional mechanical tolerance allowances, which result from the production process. In this context one would omit the application of color filters (R, G, B) for the rain sensor area (6) and thereby increase the sensitivity for the detection of rain.
According to a variant of embodiment, the upper area for the rain sensor (6) must not necessarily contain the reflexes (8) from the inner surface of the windscreen (2.1), since the modification of the light due to rain (4) is visible on the lower light spots (9). These alone can suffice as a measurement signal and can be compared e.g. with a light quantity threshold value. If the measurement signal is greater than or equal to the threshold value, it is detected that the pane is dry. If, however, the measurement signal is below the threshold value, rain (4) is detected on the outer face (2.2) of the pane (2). The more the measurement signal falls below the threshold value, the more rain is on the pane (2). This variant of embodiment offers the possibility that the area for the rain sensor (6) can be considerably reduced.
However, by means of this the upper spot (8) from
This offers for the rain sensing simultaneously the advantage that a difference image with the previous driver assistance image can be formed, thus strongly reducing the background signal and ideally leaving only the rain sensor image of the light spots (8, 9).
Often, driver assistance cameras (1) have an infrared cut filter to reduce the spectral requirements to optics and/or to permit a better color recognition. The color filters (R, G, B) on the individual pixels of the image chips (5) used today in the infrared spectral range often again have a high transmission and thus deteriorate the color selectivity.
With a spatial overlapping of the driver assistance area (7) with the rain sensor area (6) as is shown in
If better color filters (R, G, B) are used, which are no longer permeable to the infrared light, by a skilled selection of the color filter sample the rain sensor spots (8, 9) and the driver assistance image (7) can be recorded simultaneously and spatially overlapped.
In
However, the lighting source (3) is arranged far below the camera (1) outside the compact camera housing, entailing structural limitations and disadvantages.
When the opening angle of the lighting (usually by means of LEDs) is large enough, the lighting source (3) can also be placed within the camera (1), e.g. as is shown on a circuit board (12) of the camera system. This results in a significant integration advantage.
The relative effect between the beam (r2) reflected at the outer face (2.2) and the beam (rl) reflected at the inner face of the secondary beam (n) of the lighting source (3) turns out to be easily measurable and sufficient to reliably recognize raindrops (4) on the pane (2).
If infrared light is used for the lighting and the lighting source (3) is arranged as shown in
Also here raindrops (4) effect a stronger decoupling (t2′) of light from the pane in the area in front of the pane. Thus, from the camera (1) a reduced intensity of the partial beam (r2′) is measured, which was reflected at the outer face (2.2) of the pane (2).
In this form of embodiment, not the main beam (h) of the lighting source is used, but a secondary beam (n), which impinges on the camera (1) via reflections at the pane (2) as at least two spatially separated partial beams (r1; r2; r2′). Otherwise the beam paths and portions are comparable to those shown in
The mode of detection shown in
In addition, with this arrangement also the light (rh) of the main beam (h) reflected at the raindrop (4) can be used to detect rain. This is shown in
As is shown in
Of course, also both modes of detection (from
Number | Date | Country | Kind |
---|---|---|---|
10 2010 052 968 | Nov 2010 | DE | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/DE2011/001749 | 9/20/2011 | WO | 00 | 5/15/2013 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2012/092911 | 7/12/2012 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
4515443 | Bly | May 1985 | A |
4741605 | Alfredsson et al. | May 1988 | A |
5923027 | Stam et al. | Jul 1999 | A |
5987152 | Weisser | Nov 1999 | A |
6137529 | Kunimitsu et al. | Oct 2000 | A |
6331819 | Hog | Dec 2001 | B1 |
6376824 | Michenfelder et al. | Apr 2002 | B1 |
6392218 | Kuehnle | May 2002 | B1 |
6452148 | Bendicks et al. | Sep 2002 | B1 |
6555804 | Blasing | Apr 2003 | B1 |
6614015 | Ba et al. | Sep 2003 | B1 |
6617564 | Ockerse et al. | Sep 2003 | B2 |
6841767 | Mindl et al. | Jan 2005 | B2 |
6968073 | O'Boyle et al. | Nov 2005 | B1 |
7208962 | Sun et al. | Apr 2007 | B2 |
7253898 | Saikalis et al. | Aug 2007 | B2 |
7259367 | Reime | Aug 2007 | B2 |
7609857 | Franz | Oct 2009 | B2 |
7612356 | Utida et al. | Nov 2009 | B2 |
7646889 | Tsukamoto | Jan 2010 | B2 |
7855353 | Blaesing et al. | Dec 2010 | B2 |
7863568 | Fleury | Jan 2011 | B2 |
8274562 | Walter et al. | Sep 2012 | B2 |
8541732 | Rothenhaeusler | Sep 2013 | B2 |
8913132 | Seger et al. | Dec 2014 | B2 |
20020003571 | Schofield et al. | Jan 2002 | A1 |
20020081029 | Marugame | Jun 2002 | A1 |
20020148987 | Hochstein | Oct 2002 | A1 |
20030066955 | Schaub et al. | Apr 2003 | A1 |
20030201380 | Ockerse et al. | Oct 2003 | A1 |
20040004456 | LeBa et al. | Jan 2004 | A1 |
20040164981 | Fujita et al. | Aug 2004 | A1 |
20040165749 | Holz et al. | Aug 2004 | A1 |
20050035926 | Takenaga et al. | Feb 2005 | A1 |
20050063071 | Wang et al. | Mar 2005 | A1 |
20050178954 | Yukawa | Aug 2005 | A1 |
20050206511 | Heenan et al. | Sep 2005 | A1 |
20050231725 | Franz | Oct 2005 | A1 |
20050254688 | Franz | Nov 2005 | A1 |
20060076477 | Ishikawa | Apr 2006 | A1 |
20060163458 | Reime | Jul 2006 | A1 |
20060191215 | Stark | Aug 2006 | A1 |
20070075220 | Kotani | Apr 2007 | A1 |
20070216768 | Smith et al. | Sep 2007 | A1 |
20070267993 | Leleve et al. | Nov 2007 | A1 |
20070268470 | Shibazaki | Nov 2007 | A1 |
20080027607 | Ertl et al. | Jan 2008 | A1 |
20080049344 | DeWard et al. | Feb 2008 | A1 |
20080185603 | Itoi et al. | Aug 2008 | A1 |
20080265134 | Kinoshita | Oct 2008 | A1 |
20080296577 | Yuan et al. | Dec 2008 | A1 |
20090085755 | Schafer et al. | Apr 2009 | A1 |
20090128629 | Egbert et al. | May 2009 | A1 |
20090201366 | Sase et al. | Aug 2009 | A1 |
20100208060 | Kobayashi et al. | Aug 2010 | A1 |
20110031921 | Han | Feb 2011 | A1 |
20110043624 | Haug | Feb 2011 | A1 |
20110128543 | Choi | Jun 2011 | A1 |
20110204206 | Taoka | Aug 2011 | A1 |
20110253917 | Rothenhaeusler | Oct 2011 | A1 |
20110273564 | Seger et al. | Nov 2011 | A1 |
20120026318 | Huelsen et al. | Feb 2012 | A1 |
20120026330 | Huelsen et al. | Feb 2012 | A1 |
20120153154 | Rothenhaeusler et al. | Jun 2012 | A1 |
20140321709 | Kasahara et al. | Oct 2014 | A1 |
20150034827 | Kroekel | Feb 2015 | A1 |
20150070499 | Roelke et al. | Mar 2015 | A1 |
Number | Date | Country |
---|---|---|
1657905 | Aug 2005 | CN |
101896375 | Nov 2010 | CN |
44 17 385 | Nov 1995 | DE |
195 04 606 | Aug 1996 | DE |
197 04 818 | Aug 1997 | DE |
202 07 170 | Aug 2002 | DE |
102 30 200 | Jan 2004 | DE |
197 00 665 | Jul 2004 | DE |
103 03 046 | Oct 2004 | DE |
103 16 794 | Nov 2004 | DE |
103 22 010 | Dec 2004 | DE |
103 55 205 | Jul 2005 | DE |
EP 1580092 | Sep 2005 | DE |
102004015040 | Oct 2005 | DE |
102004037871 | Mar 2006 | DE |
102005004513 | Mar 2006 | DE |
102006008274 | Aug 2007 | DE |
102006010671 | Sep 2007 | DE |
102006022404 | Nov 2007 | DE |
102007061725 | Jun 2009 | DE |
102008043737 | May 2010 | DE |
102008062977 | Jun 2010 | DE |
102009000003 | Jul 2010 | DE |
102009000004 | Jul 2010 | DE |
102009000005 | Jul 2010 | DE |
0 832 798 | Apr 1998 | EP |
1 580 092 | Sep 2005 | EP |
1 764 835 | Mar 2007 | EP |
1 923 695 | May 2008 | EP |
S57-004133 | Jan 1982 | JP |
S60-125260 | Aug 1985 | JP |
H04-061379 | Feb 1992 | JP |
H11-234474 | Aug 1999 | JP |
2003-315256 | Nov 2003 | JP |
2005-292544 | Oct 2005 | JP |
2006-184844 | Jul 2006 | JP |
2007-309655 | Nov 2007 | JP |
2009-092453 | Apr 2009 | JP |
2009-098477 | May 2009 | JP |
2010-096604 | Apr 2010 | JP |
2010096604 | Apr 2010 | JP |
WO 03029757 | Apr 2003 | WO |
WO 2005075248 | Aug 2005 | WO |
WO 2006015905 | Feb 2006 | WO |
WO 2006024247 | Mar 2006 | WO |
WO 2006121954 | Nov 2006 | WO |
WO 2009020918 | Feb 2009 | WO |
WO 2010072198 | Jul 2010 | WO |
WO 2010076064 | Jul 2010 | WO |
WO 2012163341 | Dec 2012 | WO |
Entry |
---|
Partial English translation of Japanese Office Action in Japanese Patent Application No. 2013-542369, mailed Jan. 7, 2015, 1 page. |
International Search Report of the International Searching Authority for International Application PCT/DE2011/001749, mailed Mar. 29, 2012, 4 pages, European Patent Office, HV Rijswijk, Netherlands. |
PCT International Preliminary Report on Patentability including English Translation of PCT Written Opinion of the International Searching Authority for International Application PCT/DE2011/001749, issued Jun. 4, 2013, 6 pages, International Bureau of WIPO, Geneva, Switzerland. |
Chinese Office Action in Chinese Patent Application No. 201180056008.4, mailed Dec. 14, 2015, 10 pages, with English translation, 12 pages. |
Number | Date | Country | |
---|---|---|---|
20130235381 A1 | Sep 2013 | US |