Patent Grant
11938795
The present invention relates generally to a vehicle vision system for a vehicle and, more particularly, to a vehicle vision system that utilizes one or more cameras at a vehicle.
Under normal driving conditions it is often the case that a vehicle driver is dazzled or bothered by an oncoming vehicle's headlights, particularly when the oncoming vehicle has its bright lights on. While a sun visor may help to shield the driver's eyes from glare from the sun or elevated light source, it does not provide glare relief to the driver for glare from lights of oncoming vehicles.
The present invention provides a driver assistance system or vision system or imaging system for a vehicle that provides a dynamic blocking or attenuating of headlights of oncoming vehicles or other bright lights that may be ahead of the subject vehicle and that may be bothersome to the driver of the subject vehicle.
To avoid the subject vehicle driver from being bothered or dazzled or temporarily blinded by such external light sources ahead of the vehicle, the vehicle windshield can create black spots or darkened spots or reduced light-transmitting regions. With these darkened spots or reduced light-transmitting regions, the bright light emitted by the light sources will be covered or attenuated so that the driver is no longer dazzled or temporarily blinded by the lights. These darkened spots or reduced light-transmitting regions may move to follow the lights dynamically and will change his size as needed. Also, the intensity or darkness or transmissivity of each spot or reduced light-transmitting region can change dynamically so that it corresponds with the intensity of the external light. The spots or reduced light-transmitting regions may also have smooth gradient border transitions from black (or dark) to transparent to avoid a hard dark-transparent contrast.
These and other objects, advantages, purposes and features of the present invention will become apparent upon review of the following specification in conjunction with the drawings.
Referring now to the drawings and the illustrative embodiments depicted therein, a vision system 10 for a vehicle 12 includes at least one exterior viewing imaging sensor or camera 14, such as a forward viewing imaging sensor or camera, which may be disposed at and behind the windshield 16 of the vehicle and viewing forward through the windshield so as to capture image data representative of the scene occurring forward of the vehicle (
The windshield 16 comprises an active partially dimmable windshield, which may comprise an electro-optic medium (such as an electrochromic medium) sandwiched between two glass sheets or panels, or may comprise a locally dimmable liquid crystal window construction or the like. The control 20, responsive to processing by a processor of image data captured by the forward viewing camera 14, determines presence of a light source ahead of the vehicle and, responsive to processing by a processor of image data captured by the in-cabin camera 18, determines the driver's eye location and/or gaze direction. The control determines a viewing path between the driver's eyes and the detected light source and controls the windshield so as to locally dim or darken a portion (or portions) of the windshield (while the non-darkened rest of the windshield remains transparent or light-transmitting) where the determined viewing path passes through the windshield, such that the locally dimmed or darkened portions or regions are partially non-light-transmitting so as to block or attenuate or dim the bright light in the driver's view.
With the help of the forward viewing or front facing camera 14, the upcoming light sources are be detected. The system, via processing of image processing of image data captured by the camera 14, determines the size, brightness/intensity and location or position, in real Cartesian coordinates (or any coordinate system or reference system that aids in determining location of the light source relative to the vehicle and/or driver's eyes), of the detected light spots ahead of the vehicle (see, for example,
Concurrently with determination of size, intensity and location of the light sources, the in-cabin camera 18 will detect the human eyes of the driver and, via processing of image data captured by the camera 18, the system determines the driver's viewing direction or gaze direction. This information is also communicated to or provided to the ECU of the active partially dimmable windshield.
Based on this two information's from the forward viewing camera 14 and the in-cabin camera 18, the ECU of the active partially dimmable windshield determines if the driver is dazzled (which may be determined at least in part to the relative location of the detected light source to the driver's eyes and the determined intensity of the detected light source). For example, the control may determine that a windshield region for a detected light source should be darkened when it determines that the light source has a brightness or intensity of, for example, greater than 300 lux, or greater than 500 lux or greater than 1,000 lux or more, depending on the particular application. If the intensity increases (such as if the oncoming vehicle headlights are switched from a low beam setting to a high beam setting), the system may increase the darkening of the respective windshield region to reduce or avoid dazzling of the driver of the vehicle.
If the ECU or system determines that the driver is dazzled by a particular light source, the ECU will (based on the location of the light source and the driver's eyes relative to one another and relative to the windshield) calculate the position at the windshield where the driver's viewing path (between the driver's eyes and the particular light source) passes through the windshield and will adjust the windshield to cover or darken these positions with a darkened region or dark spot (see, for example,
The system and active partially dimmable windshield will not only operate to darken or reduce visibility of or glare from light sources of approaching vehicles. The system and active partially dimmable windshield can also be used to provide glare relief to the driver for all glaring light sources, such as street lights or low sun conditions (see, for example,
Optionally, and such as for a sunrise or sunset condition where the sun provides a large bright light ahead of the vehicle, the system may provide a wide darkened region to cover more than just the spot of the sun, but also high intensity light across the windshield area. For example, and with reference to
Referring now to
Optionally, the system may include multiple exterior viewing imaging sensors or cameras, such as a forward viewing camera at the front of the vehicle, and a sideward/rearward viewing camera at respective sides of the vehicle, and a rearward viewing camera at the rear of the vehicle, which capture images exterior of the vehicle. The camera or cameras each include a lens for focusing images at or onto an imaging array or imaging plane or imager of the camera. Optionally, the forward viewing camera 114 disposed at the windshield of the vehicle and viewing through the windshield and forward of the vehicle may capture image data for a machine vision system or driving assist system (such as for traffic sign recognition, headlamp control, pedestrian detection, collision avoidance, lane marker detection and/or the like). The vision system 110 includes a control or electronic control unit (ECU) 120 having a data processor and associated circuitry and software, with the data processor being operable to process image data captured by the light sensor and/or camera or cameras, whereby the ECU may detect or determine presence of objects or the like and/or the system provide displayed images at a display device for viewing by the driver of the vehicle. The data transfer or signal communication from the camera to the ECU may comprise any suitable data or communication link, such as a vehicle network bus or the like of the equipped vehicle.
The windshield 116 comprises a controlled surface or active partially dimmable windshield, which may comprise an electro-optic medium (such as an electrochromic medium) sandwiched between two glass sheets or panels, or may comprise a locally dimmable liquid crystal window construction or the like. The control 120, responsive to processing by a processor of data captured by the forward sensing light sensor 122 (and/or processing of image data captured by the forward viewing camera 114), determines presence of light sources ahead of the vehicle and, responsive to processing by a processor of image data captured by the in-cabin camera 118, determines the driver's eye location and/or gaze direction. The control determines a light path between the detected light source and the driver's eyes and controls the windshield so as to locally dim or darken a portion (or portions) of the windshield (while the non-darkened rest of the windshield remains transparent or light-transmitting) where the determined viewing path passes through the windshield, such that the locally dimmed or darkened portions or regions are partially non-light-transmitting so as to block or attenuate or dim the bright light in the driver's view.
Flash blindness is a temporary visual impairment of the driver when the driver's eyes are exposed to high intensity light ahead of the vehicle. The symptoms might last from few seconds to multiple minutes. During that time an operator may be unable to react to variety of situations. The controlled surface or windshield comprises a transparent material that is capable of changing its degree of transparency or light transmissivity in different regions at the windshield. The controlled surface may be a variably transmissive device or element that is disposed at the windshield (such as at the in-cabin surface of the vehicle windshield) or may be part of the windshield itself (whereby the windshield comprises a laminate windshield having an electro-optic or electrochromic material or medium disposed between two glass sheets).
To prevent flash blindness, the controlled surface can be used to allow the system to temporarily change transparency of the controlled surface and/or windshield to alleviate the effect of flash blindness. As shown in
With that position or location at the controlled surface determined, the microcontroller sends a signal to the control surface to lower the transparency at that region or location. For example, the controller may operate to reduce the visible light transmissivity of the control surface at a particular coordinate (X, Y) of the control surface (such as at a center point of the light path between the light source and the driver's eyes, with radius (R) around that coordinate (with R being selected depending on the size and intensity of the light source, with at a particular degree of attenuation (D), with the degree of attenuation being determined based on the intensity of light emitted by the light source and the intensity of other light sources detected ahead of the vehicle. Thus, the output of the controller may provide values for X, Y, R, D, whereby the control surface reduces visible light transmissivity at the appropriate location and size region. The process loops infinitely during normal operation of the vehicle, such as when the vehicle is being driven forward along a road.
The system thus determines the location at the windshield where light from a bright light source ahead of the vehicle will pass along a light path from the light source to the driver's eyes. The system determines the driver's eye location relative to the windshield via processing of data captured by the eye position sensor or driver monitoring system, and determines the location of the light source relative to the windshield via processing of data captured by the forward viewing camera and/or light position sensor. Based on the determined relative locations of the driver's eyes and light source relative to the windshield, the system can determine where the light path will intersect and pass through the windshield (taking into account the angle of the windshield). The system controls or adjusts the light transmissivity of the windshield (or control surface) at that location, and adjusts the light transmissivity of the windshield (or control surface) at a region around that point, with the size or radius of the region being determined based on light intensity and size of the light source. The system controls or adjusts the degree of the light transmissivity of the darkened region responsive to the determined intensity of the light emitted by the light source.
For example, the control may determine that a windshield region for a detected light source should be darkened when it determines that the light source has a brightness or intensity of, for example, greater than 300 lux, or greater than 500 lux or greater than 1,000 lux or more, depending on the particular application. If the intensity increases (such as if the oncoming vehicle headlights are switched from a low beam setting to a high beam setting), the system may increase the darkening of the respective windshield region to reduce or avoid dazzling of the driver of the vehicle.
If the ECU or system determines that the driver is dazzled by a particular light source, the ECU will (based on the location of the light source and the driver's eyes relative to one another and relative to the windshield) calculate the position at the windshield where the driver's viewing path (between the driver's eyes and the particular light source) passes through the windshield and will adjust the windshield to cover or darken these positions with a darkened region or dark spot or lower visible light transmissivity. Such calculation may take into account the distance of the driver's eyes from the windshield and the angle or slant of the windshield and a determined or estimated distance to the detected light source.
The threshold level of light intensity of the detected light source at which the control determines whether or not to reduce visible light transmissivity at the controllable transparent element may vary responsive to the determined ambient light level (e.g., a bright light in darkness ahead of the vehicle will be much more likely to cause flash blindness than the same light on a sunny day) and responsive to other detected light sources (e.g., a bright light when by itself in darkness ahead of the vehicle will be much more likely to cause flash blindness than the same light when there are multiple lights also emitting light ahead of the vehicle, such as in city driving conditions).
Thus, the threshold level dynamically varies with the determined ambient light at or ahead of the vehicle and/or with the determined activity level of light (e.g., the number of other light sources determined ahead of the vehicle). The ambient light level may be sensed or determined via a separate ambient light sensor or via the forward viewing camera. Thus, during nighttime or lower ambient lighting driving conditions, the threshold level at which the system locally darkens the windshield or controllable element is lower than the threshold level at which the system locally darkens the windshield or element during daytime or higher ambient lighting driving conditions.
The system may be scaled depending on the particular applicant. For example, a smaller version might be implemented in eye glasses worn by a user, and a larger version can be implemented into or on the windshield of a vehicle. Optionally, the system may be implemented in goggles or glasses worn by pilots, such as for aviation applications. Optionally, the system may be implemented in goggles or glasses or face shields of astronauts to avoid temporary blindness during a spacewalk. While many new vehicles today are equipped with automatic high beam function (where the high beams of the vehicle headlights are automatically switched to low beams upon detection of a vehicle ahead), such a function or feature does not avoid flash blindness completely. There are many other light sources on the road that pose the danger, such as, for example, street lights, brake lights, sun reflections from snow and/or the like. Also in certain weather conditions, a driver may be bothered by a halo effect at the windshield, which blurs the source of the light.
The controllable transparent element or windshield is electrically variable to provide the selected darkened regions at the determined locations. The controllable transparent element or windshield may have multiple regions that are separately and independently energized (with electrically conductive layers at the glass sheets that are electrically isolated to provide the independently energizable regions) to provide the darkened regions (where one or more regions may be energized to provide the appropriate size and location of the darkened spot or spots). The degree of darkening of each darkened region may be variable and may be adjusted responsive to a determined intensity of the respective detected light source (and may darken more as the light intensity increases). The controllable transparent element or windshield or system may utilize aspects of the systems described in U.S. Publication Nos. US-2020-0296273 and/or US-2011-0273659, which are hereby incorporated herein by reference in their entireties. The windshield may comprise an electro-optic construction or a liquid crystal construction or the like (or any suitable construction that provides local dimming of regions of the windshield while not dimming other regions), such as by utilizing aspects of the assemblies and devices described in U.S. Pat. Nos. 10,227,046; 7,310,178; 7,274,501; 7,255,451; 7,195,381; 7,184,190; 6,690,268; 6,859,245; 6,784,956; 6,717,639; 6,449,082; 6,144,430; 5,841,496; 5,724,187; 5,668,663; 5,406,414 and/or 5,066,112, which are hereby incorporated herein by reference in their entireties.
The camera comprises a forward viewing camera, and may be disposed at a windshield electronics module (WEM) or the like. The forward viewing camera may utilize aspects of the systems described in U.S. Pat. Nos. 9,896,039; 9,871,971; 9,596,387; 9,487,159; 8,256,821; 7,480,149; 6,824,281 and/or 6,690,268, and/or U.S. Publication Nos. US-2015-0327398; US-2015-0015713; US-2014-0160284; US-2014-0226012 and/or US-2009-0295181, which are all hereby incorporated herein by reference in their entireties.
The driver monitoring system/camera may utilize aspects of head and face direction and position tracking systems and/or eye tracking systems and/or gesture recognition systems. Such head and face direction and/or position tracking systems and/or eye tracking systems and/or gesture recognition systems may utilize aspects of the systems described in U.S. Pat. No. 7,914,187 and/or Publication Nos. US-2016-0137126; US-2015-0352953; US-2015-0296135; US-2015-0294169; US-2015-0232030; US-2015-0022664; US-2015-0015710; US-2015-0009010 and/or US-2014-0336876, which are hereby incorporated herein by reference in their entireties.
The system includes an image processor operable to process image data captured by the camera or cameras, such as for detecting objects or other vehicles or pedestrians or the like in the field of view of one or more of the cameras. For example, the image processor may comprise an image processing chip selected from the EYEQ family of image processing chips available from Mobileye Vision Technologies Ltd. of Jerusalem, Israel, and may include object detection software (such as the types described in U.S. Pat. Nos. 7,855,755; 7,720,580 and/or 7,038,577, which are hereby incorporated herein by reference in their entireties), and may analyze image data to detect vehicles and/or other objects. Responsive to such image processing, and when an object or other vehicle is detected, the system may generate an alert to the driver of the vehicle and/or may generate an overlay at the displayed image to highlight or enhance display of the detected object or vehicle, in order to enhance the driver's awareness of the detected object or vehicle or hazardous condition during a driving maneuver of the equipped vehicle.
The imaging sensors or cameras may capture image data for image processing and may comprise any suitable camera or sensing device, such as, for example, a two dimensional CMOS imaging array of a plurality of photosensor elements arranged in at least 640 columns and 480 rows (at least a 640×480 imaging array, such as a megapixel imaging array or the like), with a respective lens focusing images onto respective portions of the array. The photosensor array may comprise a plurality of photosensor elements arranged in a photosensor array having rows and columns. Preferably, the imaging array has at least 300,000 photosensor elements or pixels, more preferably at least 500,000 photosensor elements or pixels and more preferably at least 1 million photosensor elements or pixels. The imaging array may capture color image data, such as via spectral filtering at the array, such as via an RGB (red, green and blue) filter or via a red/red complement filter or such as via an RCC (red, clear, clear) filter or the like. The logic and control circuit of the imaging sensor may function in any known manner, and the image processing and algorithmic processing may comprise any suitable means for processing the images and/or image data.
For example, the vision system and/or processing and/or camera(s) and/or circuitry may utilize aspects described in U.S. Pat. Nos. 9,233,641; 9,146,898; 9,174,574; 9,090,234; 9,077,098; 8,818,042; 8,886,401; 9,077,962; 9,068,390; 9,140,789; 9,092,986; 9,205,776; 8,917,169; 8,694,224; 7,005,974; 5,760,962; 5,877,897; 5,796,094; 5,949,331; 6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202; 6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452; 6,822,563; 6,891,563; 6,946,978; 7,859,565; 5,550,677; 5,670,935; 6,636,258; 7,145,519; 7,161,616; 7,230,640; 7,248,283; 7,295,229; 7,301,466; 7,592,928; 7,881,496; 7,720,580; 7,038,577; 6,882,287; 5,929,786 and/or 5,786,772, and/or U.S. Publication Nos. US-2014-0340510; US-2014-0313339; US-2014-0347486; US-2014-0320658; US-2014-0336876; US-2014-0307095; US-2014-0327774; US-2014-0327772; US-2014-0320636; US-2014-0293057; US-2014-0309884; US-2014-0226012; US-2014-0293042; US-2014-0218535; US-2014-0218535; US-2014-0247354; US-2014-0247355; US-2014-0247352; US-2014-0232869; US-2014-0211009; US-2014-0160276; US-2014-0168437; US-2014-0168415; US-2014-0160291; US-2014-0152825; US-2014-0139676; US-2014-0138140; US-2014-0104426; US-2014-0098229; US-2014-0085472; US-2014-0067206; US-2014-0049646; US-2014-0052340; US-2014-0025240; US-2014-0028852; US-2014-005907; US-2013-0314503; US-2013-0298866; US-2013-0222593; US-2013-0300869; US-2013-0278769; US-2013-0258077; US-2013-0258077; US-2013-0242099; US-2013-0215271; US-2013-0141578 and/or US-2013-0002873, which are all hereby incorporated herein by reference in their entireties. The system may communicate with other communication systems via any suitable means, such as by utilizing aspects of the systems described in U.S. Pat. Nos. 10,071,687; 9,900,490; 9,126,525 and/or 9,036,026, which are hereby incorporated herein by reference in their entireties.
Changes and modifications in the specifically described embodiments can be carried out without departing from the principles of the invention, which is intended to be limited only by the scope of the appended claims, as interpreted according to the principles of patent law including the doctrine of equivalents.
The present application claims priority of U.S. provisional application Ser. No. 62/933,539, filed Nov. 11, 2019, and U.S. provisional application Ser. No. 62/916,962, filed Oct. 18, 2019, which are hereby incorporated herein by reference in their entireties.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5066112 | Lynam et al. | Nov 1991 | A |
| 5406414 | O'Farrell et al. | Apr 1995 | A |
| 5550677 | Schofield et al. | Aug 1996 | A |
| 5668663 | Varaprasad et al. | Sep 1997 | A |
| 5670935 | Schofield et al. | Sep 1997 | A |
| 5724187 | Varaprasad et al. | Mar 1998 | A |
| 5841496 | Itoh et al. | Nov 1998 | A |
| 5949331 | Schofield et al. | Sep 1999 | A |
| 6144430 | Kuo | Nov 2000 | A |
| 6449082 | Agrawal et al. | Sep 2002 | B1 |
| 6690268 | Schofield et al. | Feb 2004 | B2 |
| 6717639 | Jung | Apr 2004 | B2 |
| 6784956 | Matsumoto et al. | Aug 2004 | B2 |
| 6824281 | Schofield et al. | Nov 2004 | B2 |
| 6859245 | Faris et al. | Feb 2005 | B2 |
| 6995891 | Agrawal et al. | Feb 2006 | B2 |
| 7038577 | Pawlicki et al. | May 2006 | B2 |
| 7184190 | McCabe et al. | Feb 2007 | B2 |
| 7195381 | Lynam et al. | Mar 2007 | B2 |
| 7255451 | McCabe et al. | Aug 2007 | B2 |
| 7274501 | McCabe et al. | Sep 2007 | B2 |
| 7310178 | Lawlor et al. | Dec 2007 | B2 |
| 7480149 | DeWard et al. | Jan 2009 | B2 |
| 7720580 | Higgins-Luthman | May 2010 | B2 |
| 7855755 | Weller et al. | Dec 2010 | B2 |
| 7914187 | Higgins-Luthman et al. | Mar 2011 | B2 |
| 8256821 | Lawlor et al. | Sep 2012 | B2 |
| 9487159 | Achenbach | Nov 2016 | B2 |
| 9555803 | Pawlicki | Jan 2017 | B2 |
| 9596387 | Achenbach et al. | Mar 2017 | B2 |
| 9871971 | Wang et al. | Jan 2018 | B2 |
| 9896039 | Achenbach et al. | Feb 2018 | B2 |
| 10227046 | Habibi | Mar 2019 | B2 |
| 10319272 | Carter | Jun 2019 | B1 |
| 11082630 | Staab et al. | Aug 2021 | B2 |
| 11130391 | Choi | Sep 2021 | B2 |
| 20090295181 | Lawlor et al. | Dec 2009 | A1 |
| 20110204209 | Barrows | Aug 2011 | A1 |
| 20110273659 | Sobecki | Nov 2011 | A1 |
| 20140160284 | Achenbach et al. | Jun 2014 | A1 |
| 20140226012 | Achenbach | Aug 2014 | A1 |
| 20140336876 | Gieseke et al. | Nov 2014 | A1 |
| 20150009010 | Biemer | Jan 2015 | A1 |
| 20150015710 | Tiryaki | Jan 2015 | A1 |
| 20150015713 | Wang et al. | Jan 2015 | A1 |
| 20150022664 | Pflug et al. | Jan 2015 | A1 |
| 20150077826 | Beckman | Mar 2015 | A1 |
| 20150219809 | Ruhnau et al. | Aug 2015 | A1 |
| 20150232030 | Bongwald | Aug 2015 | A1 |
| 20150294169 | Zhou et al. | Oct 2015 | A1 |
| 20150296135 | Wacquant et al. | Oct 2015 | A1 |
| 20150327398 | Achenbach et al. | Nov 2015 | A1 |
| 20150352953 | Koravadi | Dec 2015 | A1 |
| 20160137126 | Fursich et al. | May 2016 | A1 |
| 20160288699 | Solar | Oct 2016 | A1 |
| 20170177000 | Meyhofer et al. | Jun 2017 | A1 |
| 20180111451 | Martens | Apr 2018 | A1 |
| 20180126907 | Jeromin | May 2018 | A1 |
| 20190168586 | Paepcke | Jun 2019 | A1 |
| 20190179320 | Pacala | Jun 2019 | A1 |
| 20190217780 | Yamaguchi | Jul 2019 | A1 |
| 20190244040 | Herrmann | Aug 2019 | A1 |
| 20200101906 | Kobayashi | Apr 2020 | A1 |
| 20210114439 | Moschek | Apr 2021 | A1 |
| 20210237651 | Larson | Aug 2021 | A1 |
| Number | Date | Country |
|---|---|---|
| 6519355 | May 2019 | JP |
| Number | Date | Country | |
|---|---|---|---|
| 20210114439 A1 | Apr 2021 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62933539 | Nov 2019 | US | |
| 62916962 | Oct 2019 | US |
