The present invention relates generally to a vehicle vision system for a vehicle and, more particularly, to a vehicle vision system that utilizes one or more cameras at a vehicle.
Use of imaging sensors in vehicle imaging systems is common and known. Examples of such known systems are described in U.S. Pat. Nos. 5,949,331; 5,670,935 and/or 5,550,677, which are hereby herein by reference in their entireties. It is also known to provide a driving assistance system that assists in driving a vehicle along a road. Examples of such systems are described in U.S. Pat. Nos. 10,115,314; 9,682,712 and 6,842,687, and/or Indian Patent Application 201741046955, published as Publication No. 27/2019 on Jul. 5, 2019 for “A Method and System for Providing Overtaking Assistance,” which are hereby incorporated herein by reference in their entireties.
Implementations herein provide a vehicular control system that includes a camera disposed at a vehicle equipped with the vehicular control system and viewing at least forward of the vehicle. The camera captures image data. The camera includes a CMOS imaging array with at least one million photosensors arranged in rows and columns. The system includes an electronic control unit (ECU) with electronic circuitry and associated software. The electronic circuitry of the ECU includes an image processor for processing image data captured by the camera to detect presence of an object viewed by the camera. The vehicular control system, as the equipped vehicle travels along a traffic lane of a road, and responsive to processing by the image processor of image data captured by the camera, determines presence of a leading vehicle ahead of the equipped vehicle and travelling in the same traffic lane as the equipped vehicle. The vehicular control system, via processing by the image processor of image data captured by the camera, determines a visual indication provided by the determined leading vehicle indicative of a safeness of a passing maneuver by the equipped vehicle. The visual indication is representative of a determination by the leading vehicle that the safeness of the passing maneuver exceeds a threshold safeness level. Based on the determined visual indication, the vehicular control system at least in part controls operation of the equipped vehicle to overtake and pass the determined leading vehicle.
These and other objects, advantages, purposes and features of the present invention will become apparent upon review of the following specification in conjunction with the drawings.
A vehicle vision system and/or driver or driving assist system and/or object detection system and/or alert system operates to capture images exterior of the vehicle and may process the captured image data to display images and to detect objects at or near the vehicle and in the predicted path of the vehicle, such as to assist a driver of the vehicle in maneuvering the vehicle in a forward or rearward direction. The vision system includes an image processor or image processing system that is operable to receive image data from one or more cameras and provide an output to a display device for displaying images representative of the captured image data. Optionally, the vision system may provide display, such as a rearview display or a top down or bird's eye or surround view display or the like.
Referring now to the drawings and the illustrative embodiments depicted therein, a vehicle 10 includes an imaging system or vision system 12 that includes at least one exterior viewing imaging sensor or camera, such as a rearward viewing imaging sensor or camera 14a (and the system may optionally include multiple exterior viewing imaging sensors or cameras, such as a forward viewing camera 14b at the front (or at the windshield) of the vehicle, and a sideward/rearward viewing camera 14c, 14d at respective sides of the vehicle), which captures images exterior of the vehicle, with the camera having a lens for focusing images at or onto an imaging array or imaging plane or imager of the camera (
An overtake is a driving maneuver when a trailing vehicle (the “overtaking vehicle”) passes another vehicle (the “overtaken vehicle” or “the leading vehicle”) travelling in front of and in the same direction as the overtaking vehicle, with the passing or overtaking vehicle passing the other or overtaken vehicle in an adjacent traffic lane that is for vehicles traveling in the opposite direction. Vehicle overtaking is one of the leading causes of accidents. Due to the dynamic nature of traffic, it is typically difficult, especially at high speeds, to obtain order in every traffic situation. However, overtaking is often necessary depending on factors such as a travel plan, speed of the surrounding vehicles, etc.
Typically, an overtake or overtaking maneuver or passing maneuver is a complicated maneuver because the destination of the overtaking vehicle is not visible (e.g., the leading vehicle blocks the view of the overtaking vehicle). On-coming traffic and additional road elements such as humans and animals entering the road may not be visible to the overtaking vehicle. Implementations described herein relate to a vehicular vision system or driver assist system that automates the task of overtaking to avoid complicated movements made by human drivers in a moment of panic. The system uses sensors (e.g., cameras, radar, lidar, etc.) on a leading vehicle to detect road objects ahead and provide an indication of a safe overtake to an overtaking vehicle via vehicle-to-vehicle (V2V) communication.
The system may provide information on the oncoming road objects with V2V communication. Based on one or more forward-facing sensors (e.g. radar, one or more cameras, lidar, etc.), the leading vehicle determines the position and speed of any road objects and available space ahead of the leading vehicle and determines whether a safe overtake is possible or alternatively that an overtake would be unsafe. The leading vehicle may indicate an overtaking maneuver is safe when, for example, the leading vehicle does not detect an object (e.g., an oncoming vehicle) within a threshold distance of the leading vehicle. The safeness of the overtaking maneuver may be based on an amount of the road visible in front of the leading vehicle. For example, if a curve or hill obscures a portion of the view of the leading vehicle such that the leading vehicle cannot determine whether an object is present within the threshold distance, the leading vehicle may indicate that the overtaking maneuver is unsafe.
The leading vehicle may indicate the safety level of an overtake via a visual indication to the trailing vehicle (i.e., the overtaking vehicle). For example, the visual indication may include an LED matrix array (e.g., of a turn signal, a brake light, or other rear facing or rearward emitting light) showing a QR-Code (or other identifiable indication such as, e.g., a symbol, pattern, blinking lights, etc.) that a camera or a driver of the overtaking vehicle receives and interprets. When overtaking is safe, the vehicle may provide a first visual indication, and the system may provide a second different visual indication when overtaking is not safe. Alternatively, the leading vehicle may provide a visual indication when overtaking is safe and no visual indication when overtaking is not safe (or vice versa). When a camera or other image sensor of the overtaking vehicle detects the visual indication, the overtaking vehicle may provide a visual or audible alert to the driver of the vehicle that is indicative of the safety level of the overtaking maneuver at that time. For example, the vehicle may display an alert on a display within the vehicle that an overtaking maneuver would be safe/unsafe. The overtaking vehicle may provide an indication to the leading vehicle of the overtaking maneuver, and in turn the leading vehicle (that is being overtaken) may provide an indication (e.g., visual, audible) of the overtake maneuver to the driver of the leading vehicle.
Optionally, a local vehicular network enables the transmission of the position, speed, and orientation of any oncoming road objects from the leading vehicle to the overtaking vehicle. Optionally, the same local vehicular network may communicate the safety of the overtaking maneuver (i.e., instead of or in addition to the visual indication). The local vehicular network may be established via any conventional wireless technologies such as WIFI, BLUETOOTH, infrared, a cellular data connection, and other V2V communication technologies.
Based on the indication provided (such as visually indicated or wirelessly transmitted) by the leading vehicle and the intent of the driver of the overtaking vehicle, the overtaking maneuver may be performed autonomously using automatic lane change assistance, distance estimation, trajectory planning and other self-driving methods. For example, the driver may indicate an intent to overtake by actuating a user input (e.g., interacting with a touch screen, pressing a button, enabling a turn signal, etc.) within the vehicle and/or providing a verbal command. In autonomous vehicles, this intent may be obtained from a decision-making unit. The safest and riskiest overtaking possibilities are determined by an accuracy of the indication from the leading vehicle and the traffic dynamic. Accounting for curves, turns, road conditions, and visibility data improves the quality of the function.
Referring now to
Referring now to
Referring now to
The safety of each potential destination 32a, 32b, 32c may be based at least in part on oncoming objects (e.g., vehicles). For example, because the destination 32c is farther away and will require a longer time in the oncoming traffic lane, the destination 32c may be less safe than the other two potential destinations 32a, 32b. The system also determines the trajectories (i.e., the speed, the acceleration, the steering angles, etc.) necessary to arrive at each potential destination. The safety of each potential destination may be based in part on the required trajectory. For example, a trajectory that requires greater acceleration or sharper steering may be less safe than trajectories that requires less acceleration or gentler steering. The system may select the safest potential destination out of all of the potential destinations.
Optionally, during the overtaking maneuver, the overtaking vehicle 22 may maintain continuous communication with the leading vehicle 24 regarding information on the obstacles ahead of the leading vehicle 24. When the leading vehicle 24 is operating autonomously or semi-autonomously, the leading vehicle 24 may attempt to maintain a distance or gap between any vehicles (e.g., vehicle 28) in front of the leading vehicle to maintain a safe destination for the overtaking vehicle 24. When the leading vehicle 24 is being operated by a driver, the leading vehicle 24 may display a notice or other alert in the cabin of the vehicle requesting the driver of the leading vehicle to maintain the gap.
Referring now to
Thus, the vehicular vision system or driver assist system provides safe overtaking maneuvering by receiving a visual and/or wireless communication from a leading vehicle regarding a level of safety of an overtaking maneuver. When the leading vehicle indicates, based on sensor data regarding objects in front of the leading vehicle, that overtaking is safe, the equipped vehicle autonomously initiates an overtaking maneuver. The overtaking vehicle and the leading vehicle may communicate via visual indications and/or via V2V communication. The leading vehicle may provide the overtaking vehicle with information regarding objects detected in front of the overtaking vehicle. The overtaking vehicle, during the overtaking maneuver may determine a plurality of potential destinations and select the destination that is the safest.
For autonomous vehicles suitable for deployment with the systems described herein, an occupant of the vehicle may, under particular circumstances, be desired or required to take over operation/control of the vehicle and drive the vehicle so as to avoid potential hazard for as long as the autonomous system relinquishes such control or driving. Such occupant of the vehicle thus becomes the driver of the autonomous vehicle. As used herein, the term “driver” refers to such an occupant, even when that occupant is not actually driving the vehicle, but is situated in the vehicle so as to be able to take over control and function as the driver of the vehicle when the vehicle control system hands over control to the occupant or driver or when the vehicle control system is not operating in an autonomous or semi-autonomous mode.
The system may also communicate with other systems, such as via a vehicle-to-vehicle communication system or a vehicle-to-infrastructure communication system or the like. Such car2car or vehicle to vehicle (V2V) and vehicle-to-infrastructure (car2X or V2X or V2I or a 4G or 5G broadband cellular network) technology provides for communication between vehicles and/or infrastructure based on information provided by one or more vehicles and/or information provided by a remote server or the like. Such vehicle communication systems may utilize aspects of the systems described in U.S. Pat. Nos. 6,690,268; 6,693,517 and/or 7,580,795, and/or U.S. Publication Nos. US-2014-0375476; US-2014-0218529; US-2013-0222592; US-2012-0218412; US-2012-0062743; US-2015-0251599; US-2015-0158499; US-2015-0124096; US-2015-0352953; US-2016-0036917 and/or US-2016-0210853, which are hereby incorporated herein by reference in their entireties.
Typically an autonomous vehicle would be equipped with a suite of sensors, including multiple machine vision cameras deployed at the front, sides and rear of the vehicle, multiple radar sensors deployed at the front, sides and rear of the vehicle, and/or multiple lidar sensors deployed at the front, sides and rear of the vehicle. Typically, such an autonomous vehicle will also have wireless two way communication with other vehicles or infrastructure, such as via a car2car (V2V) or car2x communication system.
The camera or sensor may comprise any suitable camera or sensor. Optionally, the camera may comprise a “smart camera” that includes the imaging sensor array and associated circuitry and image processing circuitry and electrical connectors and the like as part of a camera module, such as by utilizing aspects of the vision systems described in U.S. Pat. Nos. 10,099,614 and/or 10,071,687, which are hereby incorporated herein by reference in their entireties.
The system includes an image processor operable to process image data captured by the camera or cameras, such as for detecting objects or other vehicles or pedestrians or the like in the field of view of one or more of the cameras. For example, the image processor may comprise an image processing chip selected from the EYEQ family of image processing chips available from Mobileye Vision Technologies Ltd. of Jerusalem, Israel, and may include object detection software (such as the types described in U.S. Pat. Nos. 7,855,755; 7,720,580 and/or 7,038,577, which are hereby incorporated herein by reference in their entireties), and may analyze image data to detect vehicles and/or other objects. Responsive to such image processing, and when an object or other vehicle is detected, the system may generate an alert to the driver of the vehicle and/or may generate an overlay at the displayed image to highlight or enhance display of the detected object or vehicle, in order to enhance the driver's awareness of the detected object or vehicle or hazardous condition during a driving maneuver of the equipped vehicle.
The vehicle may include any type of sensor or sensors, such as imaging sensors or radar sensors or lidar sensors or ultrasonic sensors or the like. The imaging sensor or camera may capture image data for image processing and may comprise any suitable camera or sensing device, such as, for example, a two dimensional array of a plurality of photosensor elements arranged in at least 640 columns and 480 rows (at least a 640×480 imaging array, such as a megapixel imaging array or the like), with a respective lens focusing images onto respective portions of the array. The photosensor array may comprise a plurality of photosensor elements arranged in a photosensor array having rows and columns. The imaging array may comprise a CMOS imaging array having at least 300,000 photosensor elements or pixels, preferably at least 500,000 photosensor elements or pixels and more preferably at least 1 million photosensor elements or pixels. The imaging array may capture color image data, such as via spectral filtering at the array, such as via an RGB (red, green and blue) filter or via a red/red complement filter or such as via an RCC (red, clear, clear) filter or the like. The logic and control circuit of the imaging sensor may function in any known manner, and the image processing and algorithmic processing may comprise any suitable means for processing the images and/or image data.
For example, the vision system and/or processing and/or camera and/or circuitry may utilize aspects described in U.S. Pat. Nos. 9,233,641; 9,146,898; 9,174,574; 9,090,234; 9,077,098; 8,818,042; 8,886,401; 9,077,962; 9,068,390; 9,140,789; 9,092,986; 9,205,776; 8,917,169; 8,694,224; 7,005,974; 5,760,962; 5,877,897; 5,796,094; 5,949,331; 6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202; 6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452; 6,822,563; 6,891,563; 6,946,978; 7,859,565; 5,550,677; 5,670,935; 6,636,258; 7,145,519; 7,161,616; 7,230,640; 7,248,283; 7,295,229; 7,301,466; 7,592,928; 7,881,496; 7,720,580; 7,038,577; 6,882,287; 5,929,786 and/or 5,786,772, and/or U.S. Publication Nos. US-2014-0340510; US-2014-0313339; US-2014-0347486; US-2014-0320658; US-2014-0336876; US-2014-0307095; US-2014-0327774; US-2014-0327772; US-2014-0320636; US-2014-0293057; US-2014-0309884; US-2014-0226012; US-2014-0293042; US-2014-0218535; US-2014-0218535; US-2014-0247354; US-2014-0247355; US-2014-0247352; US-2014-0232869; US-2014-0211009; US-2014-0160276; US-2014-0168437; US-2014-0168415; US-2014-0160291; US-2014-0152825; US-2014-0139676; US-2014-0138140; US-2014-0104426; US-2014-0098229; US-2014-0085472; US-2014-0067206; US-2014-0049646; US-2014-0052340; US-2014-0025240; US-2014-0028852; US-2014-005907; US-2013-0314503; US-2013-0298866; US-2013-0222593; US-2013-0300869; US-2013-0278769; US-2013-0258077; US-2013-0258077; US-2013-0242099; US-2013-0215271; US-2013-0141578 and/or US-2013-0002873, which are all hereby incorporated herein by reference in their entireties. The system may communicate with other communication systems via any suitable means, such as by utilizing aspects of the systems described in U.S. Pat. Nos. 10,071,687; 9,900,490; 9,126,525 and/or 9,036,026, which are hereby incorporated herein by reference in their entireties.
The system may utilize sensors, such as radar or lidar sensors or the like. The sensing system may utilize aspects of the systems described in U.S. Pat. Nos. 9,753,121; 9,689,967; 9,599,702; 9,575,160; 9,146,898; 9,036,026; 8,027,029; 8,013,780; 7,053,357; 7,408,627; 7,405,812; 7,379,163; 7,379,100; 7,375,803; 7,352,454; 7,340,077; 7,321,111; 7,310,431; 7,283,213; 7,212,663; 7,203,356; 7,176,438; 7,157,685; 6,919,549; 6,906,793; 6,876,775; 6,710,770; 6,690,354; 6,678,039; 6,674,895 and/or 6,587,186, and/or International Publication Nos. WO 2018/007995 and/or WO 2011/090484, and/or U.S. Publication Nos. US-2018-0231635; US-2018-0045812; US-2018-0015875; US-2017-0356994; US-2017-0315231; US-2017-0276788; US-2017-0254873; US-2017-0222311 and/or US-2010-0245066, which are hereby incorporated herein by reference in their entireties.
Changes and modifications in the specifically described embodiments can be carried out without departing from the principles of the invention, which is intended to be limited only by the scope of the appended claims, as interpreted according to the principles of patent law including the doctrine of equivalents.
The present application claims the filing benefits of U.S. provisional application Ser. No. 63/198,197, filed Oct. 2, 2020, which is hereby incorporated herein by reference in its entirety.
Number | Name | Date | Kind |
---|---|---|---|
5550677 | Schofield et al. | Aug 1996 | A |
5670935 | Schofield | Sep 1997 | A |
5949331 | Schofield | Sep 1999 | A |
6587186 | Bamji et al. | Jul 2003 | B2 |
6674895 | Rafii et al. | Jan 2004 | B2 |
6678039 | Charbon | Jan 2004 | B2 |
6690268 | Schofield et al. | Feb 2004 | B2 |
6690354 | Sze | Feb 2004 | B2 |
6693517 | McCarthy et al. | Feb 2004 | B2 |
6710770 | Tomasi et al. | Mar 2004 | B2 |
6842687 | Winner et al. | Jan 2005 | B2 |
6876775 | Torunoglu | Apr 2005 | B2 |
6906793 | Bamji et al. | Jun 2005 | B2 |
6919549 | Bamji et al. | Jul 2005 | B2 |
7038577 | Pawlicki et al. | May 2006 | B2 |
7053357 | Schwarte | May 2006 | B2 |
7157685 | Bamji et al. | Jan 2007 | B2 |
7176438 | Bamji et al. | Feb 2007 | B2 |
7203356 | Gokturk et al. | Apr 2007 | B2 |
7205904 | Schofield | Apr 2007 | B2 |
7212663 | Tomasi | May 2007 | B2 |
7283213 | O'Connor et al. | Oct 2007 | B2 |
7310431 | Gokturk et al. | Dec 2007 | B2 |
7321111 | Bamji et al. | Jan 2008 | B2 |
7340077 | Gokturk et al. | Mar 2008 | B2 |
7352454 | Bamji et al. | Apr 2008 | B2 |
7375803 | Bamji | May 2008 | B1 |
7379100 | Gokturk et al. | May 2008 | B2 |
7379163 | Rafii et al. | May 2008 | B2 |
7405812 | Bamji | Jul 2008 | B1 |
7408627 | Bamji et al. | Aug 2008 | B2 |
7580795 | McCarthy et al. | Aug 2009 | B2 |
7720580 | Higgins-Luthman | May 2010 | B2 |
7855755 | Weller et al. | Dec 2010 | B2 |
8013780 | Lynam | Sep 2011 | B2 |
8027029 | Lu et al. | Sep 2011 | B2 |
9036026 | Dellantoni et al. | May 2015 | B2 |
9146898 | Ihlenburg et al. | Sep 2015 | B2 |
9575160 | Davis et al. | Feb 2017 | B1 |
9599702 | Bordes et al. | Mar 2017 | B1 |
9682712 | Kubo | Jun 2017 | B2 |
9689967 | Stark et al. | Jun 2017 | B1 |
9753121 | Davis et al. | Sep 2017 | B1 |
10071687 | Ihlenburg et al. | Sep 2018 | B2 |
10099614 | Diessner | Oct 2018 | B2 |
10115314 | Boegel | Oct 2018 | B2 |
10214157 | Achenbach | Feb 2019 | B2 |
10222224 | Johnson | Mar 2019 | B2 |
10406981 | Chundrlik, Jr. | Sep 2019 | B2 |
10457209 | Byrne | Oct 2019 | B2 |
10787125 | Achenbach | Sep 2020 | B2 |
10812992 | Tran | Oct 2020 | B1 |
11017665 | Roy | May 2021 | B1 |
11454719 | Hess et al. | Sep 2022 | B2 |
11763410 | Roy | Sep 2023 | B1 |
20050179527 | Schofield | Aug 2005 | A1 |
20080192984 | Higuchi | Aug 2008 | A1 |
20100245066 | Sarioglu et al. | Sep 2010 | A1 |
20120062743 | Lynam et al. | Mar 2012 | A1 |
20120218412 | Dellantoni et al. | Aug 2012 | A1 |
20130222592 | Gieseke | Aug 2013 | A1 |
20130297387 | Michael | Nov 2013 | A1 |
20140218529 | Mahmoud et al. | Aug 2014 | A1 |
20140375476 | Johnson et al. | Dec 2014 | A1 |
20150124096 | Koravadi | May 2015 | A1 |
20150158499 | Koravadi | Jun 2015 | A1 |
20150251599 | Koravadi | Sep 2015 | A1 |
20150352953 | Koravadi | Dec 2015 | A1 |
20160036917 | Koravadi et al. | Feb 2016 | A1 |
20160159394 | Ryu | Jun 2016 | A1 |
20160210853 | Koravadi | Jul 2016 | A1 |
20170222311 | Hess et al. | Aug 2017 | A1 |
20170254873 | Koravadi | Sep 2017 | A1 |
20170276788 | Wodrich | Sep 2017 | A1 |
20170315231 | Wodrich | Nov 2017 | A1 |
20170356994 | Wodrich et al. | Dec 2017 | A1 |
20180015875 | May et al. | Jan 2018 | A1 |
20180045812 | Hess | Feb 2018 | A1 |
20180173239 | Yoon | Jun 2018 | A1 |
20180231635 | Woehlte | Aug 2018 | A1 |
20200327343 | Lund | Oct 2020 | A1 |
20210061276 | Zhang | Mar 2021 | A1 |
20210221390 | Slobodyanyuk | Jul 2021 | A1 |
20210385865 | Mueck | Dec 2021 | A1 |
20210392454 | Choi | Dec 2021 | A1 |
20220024485 | Theverapperuma | Jan 2022 | A1 |
20220097625 | Russell | Mar 2022 | A1 |
20220255223 | Tran | Aug 2022 | A1 |
Number | Date | Country |
---|---|---|
201741056955 | May 2019 | IN |
Number | Date | Country | |
---|---|---|---|
20220105941 A1 | Apr 2022 | US |
Number | Date | Country | |
---|---|---|---|
63198197 | Oct 2020 | US |