The present invention relates generally to a vision system for a vehicle and, more particularly, to a vehicle vision system that utilizes one or more cameras at a vehicle and that utilizes a communication device to communicate with other vehicles.
Use of imaging sensors in vehicle imaging systems is common and known. Examples of such known systems are described in U.S. Pat. Nos. 5,949,331; 5,670,935 and/or 5,550,677, which are hereby incorporated herein by reference in their entireties.
Recently developed Dedicated Short Range Communication (DSRC) radio technology (as known in the art) enables communications-based active safety systems. Communication links for such applications need to be reliable, high speed, low latency links that are immune to extreme weather conditions and that work reliably in high speed mobility conditions and multipath roadway environments.
The present invention provides a driver assistance system or vision system or imaging system for a vehicle that utilizes one or more cameras (preferably one or more CMOS cameras) to capture image data representative of images exterior of the vehicle, and uses a dedicated short range communication (DSRC) radio of the vehicle to communicate compressed video images to a DSRC radio of another vehicle.
The system of the present invention may be used as part of a platooning group of vehicles (driving close together along a road), where compressed video images (captured by a forward viewing camera of the lead vehicle) are communicated to following vehicles, where the compressed video images or image data may be processed (by a machine vision processor) for use in advanced driver assistance systems (ADAS) or may be displayed at a display screen of the following vehicle so the driver of the following vehicle can view the path ahead of the lead vehicle.
Optionally, the system of the present invention may be used to assist a driver in towing a trailer with a vehicle. One or more cameras of the trailer may capture video images exterior of and surrounding the trailer (or optionally inside the trailer) and a DSRC radio of the trailer may communicate the captured images to a communication link and control of the towing vehicle, whereby the captured video images may be displayed at a display screen in the towing vehicle for the driver to view during a driving maneuver (such as during a reversing maneuver). Optionally, the control of the towing vehicle may process captured and communicated image data to detect objects in the exterior field(s) of view of the trailer camera(s), and the system may generate an alert to the driver of the vehicle if a detected object is determined to be a potential hazard or obstacle during the driving maneuver.
These and other objects, advantages, purposes and features of the present invention will become apparent upon review of the following specification in conjunction with the drawings.
A vehicle vision system and/or driver assist system and/or object detection system and/or alert system operates to capture images exterior of the vehicle and may process the captured image data to display images and to detect objects at or near the vehicle and in the predicted path of the vehicle, such as to assist a driver of the vehicle in maneuvering the vehicle in a rearward direction. The vision system includes an image processor or image processing system that is operable to receive image data from one or more cameras and provide an output to a display device for displaying images representative of the captured image data. Optionally, the vision system may provide display, such as a rearview display or a top down or bird's eye or surround view display or the like.
Referring now to the drawings and the illustrative embodiments depicted therein, a vehicle 100 includes an imaging system or vision system that includes a forward viewing camera module 101 that is disposed at and views through the windshield of the vehicle and captures image data of the scene exterior and forward of the vehicle (
Even though the main purpose of the DSRC technology is designed for the safety applications in the automotive domain, it is also designed for the no-safety use one of such application could be compressed video transfer. In the U.S., channels 178, 172 and 184 are reserved for safety applications and other SCH channels could be used for non-safety applications (the layout of the DSRC spectrum in the U.S. is shown in
One use case is in a platooning vehicle situation, where vehicles will follow very close to each other and the vehicles following the lead vehicle cannot see the road path ahead (such as platooning vehicles utilizing aspects of the systems described in U.S. patent application Ser. No. 15/203,840, filed Jul. 7, 2016, now U.S. Pat. No. 10,115,314, which is hereby incorporated herein by reference in its entirety). Also, the driver in a following vehicle may only see the vehicle directly in front of his or her vehicle and cannot see the path and obstacles ahead of and in the path of the lead vehicle. In case of an emergency, if the lead vehicle does not react and/or the following vehicles do not react in time, a multi-vehicle collision may result as the following vehicles will also end up in an accident with the lead vehicle. This situation could be prevented if the drivers in the following vehicles could see the road ahead and monitor the platooning system and take necessary action if the system fails to react.
Other use cases may be for trailer backup assist. It is very difficult to back up a trailer attached to a vehicle without the backside or side view of the trailer. Ultrasonic sensors may be installed in the trailer to detect an obstacle and alert the driver. Preferably, the driver may view the surroundings or environment around the trailer during trailer backup.
The present invention provides a method and apparatus to utilize the DSRC infrastructure for the non-safety use cases such as wireless video transfer.
To solve the issue discussed above with platooning vehicles, the DSRC infrastructure may be used to transmit compressed video images captured by the lead vehicle's windshield camera, whereby the compressed video images are transmitted back to the following vehicles and the video images showing the path ahead of the lead vehicle may be displayed to the driver on a display (such as a heads up display or HUD) of each or at least some of the following vehicles. Also, the windshield camera module (having an image processor and control) mounted at the windshield of the following vehicle may process the video image data from the lead vehicle (via its image processor) to perform lane detection and obstacle detection using the video images or image data captured by the lead vehicle camera in addition to the distance measurement between the vehicles, such as shown in
The vehicle (or vehicles) 200 (
The control 206 may optionally control or communicate with other vehicle systems via a vehicle communication or network bus 212. The control may also be in communication with the display 208 via link 205 (which may be a wired or wireless communication link or may be via the vehicle communication or network bus). The control 206 processing the video image data transmitted by the lead vehicle DSRC radio acts like a redundant monitoring unit that could control the vehicle 200 following the lead vehicle in case the lead vehicle fails to react in case of an emergency situation. The driver in the vehicle following the lead vehicle may also view the displayed video images (from the lead vehicle camera) to monitor the path and take necessary action in case of emergency if the lead vehicle fails to react.
As shown in
To solve the issue discussed above with trailer backup situations, the DSRC infrastructure may be used to transmit compressed video images captured by one or more cameras mounted on the trailer and viewing the trailer's surroundings, such as shown in
Therefore, the present invention comprises a vehicle vision system that utilizes a DSRC communication to transmit or communicate compressed video image data (as captured by a camera of the vehicle) to a system or processor of another vehicle, where the system or processor of the other vehicle may display the captured images or may process the captured image data for a function of the other vehicle. Thus, the present invention provides for an enhanced view to a driver of a following vehicle in a platoon of vehicles of the road ahead of the lead vehicle. Optionally, the present invention may provide for an enhanced view to a driver of a vehicle of the surroundings of a trailer being towed by the vehicle, via use of images or image data captured by one or more cameras at the trailer.
The system may utilize sensors, such as radar or lidar sensors or the like. The sensing system may utilize aspects of the systems described in U.S. Pat. Nos. 8,027,029; 8,013,780; 6,825,455; 7,053,357; 7,408,627; 7,405,812; 7,379,163; 7,379,100; 7,375,803; 7,352,454; 7,340,077; 7,321,111; 7,310,431; 7,283,213; 7,212,663; 7,203,356; 7,176,438; 7,157,685; 6,919,549; 6,906,793; 6,876,775; 6,710,770; 6,690,354; 6,678,039; 6,674,895 and/or 6,587,186, and/or International Publication No. WO 2011/090484 and/or U.S. Publication No. US-2010-0245066 and/or U.S. provisional applications, Ser. No. 62/375,161, filed Aug. 15, 2016, Ser. No. 62/361,586, filed Jul. 13, 2016, Ser. No. 62/359,913, filed Jul. 8, 2016, Ser. No. 62/349,874, filed Jun. 14, 2016, Ser. No. 62/330,557, filed May 2, 2016, Ser. No. 62/313,279, filed Mar. 25, 2016, Ser. No. 62/303,546, filed Mar. 4, 2016, and/or Ser. No. 62/289,441, filed Feb. 1, 2016, which are hereby incorporated herein by reference in their entireties.
The system may also communicate with other systems, such as via a vehicle-to-vehicle communication system or a vehicle-to-infrastructure communication system or the like. Such car2car or vehicle to vehicle (V2V) and vehicle-to-infrastructure (car2X or V2X or V2I or 4G or 5G) technology provides for communication between vehicles and/or infrastructure based on information provided by one or more vehicles and/or information provided by a remote server or the like. Such vehicle communication systems may utilize aspects of the systems described in U.S. Pat. Nos. 6,690,268; 6,693,517 and/or 7,580,795, and/or U.S. Publication Nos. US-2014-0375476; US-2014-0218529; US-2013-0222592; US-2012-0218412; US-2012-0062743; US-2015-0251599; US-2015-0158499; US-2015-0124096; US-2015-0352953; US-2016-0036917 and/or US-2016-0210853, which are hereby incorporated herein by reference in their entireties.
The camera or sensor may comprise any suitable camera or sensor. Optionally, the camera may comprise a “smart camera” that includes the imaging sensor array and associated circuitry and image processing circuitry and electrical connectors and the like as part of a camera module, such as by utilizing aspects of the vision systems described in International Publication Nos. WO 2013/081984 and/or WO 2013/081985, which are hereby incorporated herein by reference in their entireties.
The system includes an image processor operable to process image data captured by the camera or cameras, such as for detecting objects or other vehicles or pedestrians or the like in the field of view of one or more of the cameras. For example, the image processor may comprise an image processing chip selected from the EyeQ family of image processing chips available from Mobileye Vision Technologies Ltd. of Jerusalem, Israel, and may include object detection software (such as the types described in U.S. Pat. Nos. 7,855,755; 7,720,580 and/or 7,038,577, which are hereby incorporated herein by reference in their entireties), and may analyze image data to detect vehicles and/or other objects. Responsive to such image processing, and when an object or other vehicle is detected, the system may generate an alert to the driver of the vehicle and/or may generate an overlay at the displayed image to highlight or enhance display of the detected object or vehicle, in order to enhance the driver's awareness of the detected object or vehicle or hazardous condition during a driving maneuver of the equipped vehicle.
The camera system or camera module of the present invention may utilize aspects of the systems and/or modules described in International Publication Nos. WO 2013/123161 and/or WO 2013/019795, and/or U.S. Pat. Nos. 8,256,821; 7,480,149; 7,289,037; 7,004,593; 6,824,281; 6,690,268; 6,445,287; 6,428,172; 6,420,975; 6,326,613; 6,278,377; 6,243,003; 6,250,148; 6,172,613 and/or 6,087,953, and/or U.S. Publication Nos. US-2015-0327398; US-2014-0226012 and/or US-2009-0295181, which are all hereby incorporated herein by reference in their entireties. Optionally, the vision system may include a plurality of exterior facing imaging sensors or cameras, such as a rearward facing imaging sensor or camera, a forwardly facing camera at the front of the vehicle, and sidewardly/rearwardly facing cameras at respective sides of the vehicle, which capture image data representative of the scene exterior of the vehicle.
For example, the vision system and/or processing and/or camera and/or circuitry may utilize aspects described in U.S. Pat. Nos. 9,233,641; 9,146,898; 9,174,574; 9,090,234; 9,077,098; 8,818,042; 8,886,401; 9,077,962; 9,068,390; 9,140,789; 9,092,986; 9,205,776; 8,917,169; 8,694,224; 7,005,974; 5,760,962; 5,877,897; 5,796,094; 5,949,331; 6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202; 6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452; 6,822,563; 6,891,563; 6,946,978; 7,859,565; 5,550,677; 5,670,935; 6,636,258; 7,145,519; 7,161,616; 7,230,640; 7,248,283; 7,295,229; 7,301,466; 7,592,928; 7,881,496; 7,720,580; 7,038,577; 6,882,287; 5,929,786 and/or 5,786,772, and/or U.S. Publication Nos. US-2014-0340510; US-2014-0313339; US-2014-0347486; US-2014-0320658; US-2014-0336876; US-2014-0307095; US-2014-0327774; US-2014-0327772; US-2014-0320636; US-2014-0293057; US-2014-0309884; US-2014-0226012; US-2014-0293042; US-2014-0218535; US-2014-0218535; US-2014-0247354; US-2014-0247355; US-2014-0247352; US-2014-0232869; US-2014-0211009; US-2014-0160276; US-2014-0168437; US-2014-0168415; US-2014-0160291; US-2014-0152825; US-2014-0139676; US-2014-0138140; US-2014-0104426; US-2014-0098229; US-2014-0085472; US-2014-0067206; US-2014-0049646; US-2014-0052340; US-2014-0025240; US-2014-0028852; US-2014-005907; US-2013-0314503; US-2013-0298866; US-2013-0222593; US-2013-0300869; US-2013-0278769; US-2013-0258077; US-2013-0258077; US-2013-0242099; US-2013-0215271; US-2013-0141578 and/or US-2013-0002873, which are all hereby incorporated herein by reference in their entireties. The system may communicate with other communication systems via any suitable means, such as by utilizing aspects of the systems described in International Publication Nos. WO/2010/144900; WO 2013/043661 and/or WO 2013/081985, and/or U.S. Pat. No. 9,126,525, which are hereby incorporated herein by reference in their entireties.
Optionally, the vision system may include a display for displaying images captured by one or more of the imaging sensors for viewing by the driver of the vehicle while the driver is normally operating the vehicle. Optionally, for example, the vision system may include a video display device, such as by utilizing aspects of the video display systems described in U.S. Pat. Nos. 5,530,240; 6,329,925; 7,855,755; 7,626,749; 7,581,859; 7,446,650; 7,338,177; 7,274,501; 7,255,451; 7,195,381; 7,184,190; 5,668,663; 5,724,187; 6,690,268; 7,370,983; 7,329,013; 7,308,341; 7,289,037; 7,249,860; 7,004,593; 4,546,551; 5,699,044; 4,953,305; 5,576,687; 5,632,092; 5,677,851; 5,708,410; 5,737,226; 5,802,727; 5,878,370; 6,087,953; 6,173,508; 6,222,460; 6,513,252 and/or 6,642,851, and/or U.S. Publication Nos. US-2012-0162427; US-2006-0050018 and/or US-2006-0061008, which are all hereby incorporated herein by reference in their entireties. Optionally, the vision system (utilizing the forward facing camera and a rearward facing camera and other cameras disposed at the vehicle with exterior fields of view) may be part of or may provide a display of a top-down view or bird's-eye view system of the vehicle or a surround view at the vehicle, such as by utilizing aspects of the vision systems described in International Publication Nos. WO 2010/099416; WO 2011/028686; WO 2012/075250; WO 2013/019795; WO 2012/075250; WO 2012/145822; WO 2013/081985; WO 2013/086249 and/or WO 2013/109869, and/or U.S. Publication No. US-2012-0162427, which are hereby incorporated herein by reference in their entireties.
Changes and modifications in the specifically described embodiments can be carried out without departing from the principles of the invention, which is intended to be limited only by the scope of the appended claims, as interpreted according to the principles of patent law including the doctrine of equivalents.
The present application claims the filing benefits of U.S. provisional application Ser. No. 62/263,187, filed Dec. 4, 2015, which is hereby incorporated herein by reference in its entirety.
Number | Name | Date | Kind |
---|---|---|---|
4910793 | Mainardi | Mar 1990 | A |
5550677 | Schofield et al. | Aug 1996 | A |
5670935 | Schofield et al. | Sep 1997 | A |
5760962 | Schofield et al. | Jun 1998 | A |
5786772 | Schofield et al. | Jul 1998 | A |
5796094 | Schofield et al. | Aug 1998 | A |
5877897 | Schofield et al. | Mar 1999 | A |
5929786 | Schofield et al. | Jul 1999 | A |
5949331 | Schofield et al. | Sep 1999 | A |
6201642 | Bos | Mar 2001 | B1 |
6302545 | Schofield et al. | Oct 2001 | B1 |
6396397 | Bos et al. | May 2002 | B1 |
6405132 | Breed et al. | Jun 2002 | B1 |
6498620 | Schofield et al. | Dec 2002 | B2 |
6523964 | Schofield et al. | Feb 2003 | B2 |
6587186 | Bamji et al. | Jul 2003 | B2 |
6611202 | Schofield et al. | Aug 2003 | B2 |
6636258 | Strumolo | Oct 2003 | B2 |
6674895 | Rafii et al. | Jan 2004 | B2 |
6678039 | Charbon | Jan 2004 | B2 |
6690268 | Schofield et al. | Feb 2004 | B2 |
6690354 | Sze | Feb 2004 | B2 |
6693517 | McCarthy et al. | Feb 2004 | B2 |
6710770 | Tomasi et al. | Mar 2004 | B2 |
6802617 | Schofield et al. | Oct 2004 | B2 |
6806452 | Bos et al. | Oct 2004 | B2 |
6822563 | Bos et al. | Nov 2004 | B2 |
6825455 | Schwarte | Nov 2004 | B1 |
6876775 | Torunoglu | Apr 2005 | B2 |
6882287 | Schofield | Apr 2005 | B2 |
6891563 | Schofield et al. | May 2005 | B2 |
6906793 | Bamji et al. | Jun 2005 | B2 |
6919549 | Bamji et al. | Jul 2005 | B2 |
6946978 | Schofield | Sep 2005 | B2 |
6975246 | Trudeau | Dec 2005 | B1 |
7005974 | McMahon et al. | Feb 2006 | B2 |
7038577 | Pawlicki et al. | May 2006 | B2 |
7053357 | Schwarte | May 2006 | B2 |
7145519 | Takahasi et al. | Dec 2006 | B2 |
7157685 | Bamji et al. | Jan 2007 | B2 |
7161616 | Okamoto et al. | Jan 2007 | B1 |
7176438 | Bamji et al. | Feb 2007 | B2 |
7203356 | Gokturk et al. | Apr 2007 | B2 |
7212663 | Tomasi | May 2007 | B2 |
7230640 | Regensburger et al. | Jun 2007 | B2 |
7248283 | Takagi et al. | Jul 2007 | B2 |
7283213 | O'Connor et al. | Oct 2007 | B2 |
7295229 | Kumata et al. | Nov 2007 | B2 |
7301466 | Asai | Nov 2007 | B2 |
7310431 | Gokturk et al. | Dec 2007 | B2 |
7321111 | Bamji et al. | Jan 2008 | B2 |
7340077 | Gokturk et al. | Mar 2008 | B2 |
7352454 | Bamji et al. | Apr 2008 | B2 |
7375803 | Bamji | May 2008 | B1 |
7379100 | Gokturk et al. | May 2008 | B2 |
7379163 | Rafii et al. | May 2008 | B2 |
7405812 | Bamji | Jul 2008 | B1 |
7408627 | Bamji et al. | Aug 2008 | B2 |
7580795 | McCarthy et al. | Aug 2009 | B2 |
7592928 | Chinomi et al. | Sep 2009 | B2 |
7720580 | Higgins-Luthman | May 2010 | B2 |
7855755 | Weller et al. | Dec 2010 | B2 |
7859565 | Schofield et al. | Dec 2010 | B2 |
7881496 | Camilleri et al. | Feb 2011 | B2 |
8013780 | Lynam | Sep 2011 | B2 |
8027029 | Lu et al. | Sep 2011 | B2 |
9688199 | Koravadi | Jun 2017 | B2 |
20030095039 | Shimomura et al. | May 2003 | A1 |
20060254142 | Das et al. | Nov 2006 | A1 |
20080068455 | Pratt | Mar 2008 | A1 |
20090033474 | Chen | Feb 2009 | A1 |
20100085171 | Do | Apr 2010 | A1 |
20100245066 | Sarioglu | Sep 2010 | A1 |
20110032119 | Pfeiffer et al. | Feb 2011 | A1 |
20120062743 | Lynam et al. | Mar 2012 | A1 |
20120065858 | Nickolaou et al. | Mar 2012 | A1 |
20120218412 | Dellantoni et al. | Aug 2012 | A1 |
20130116859 | Ihlenburg et al. | May 2013 | A1 |
20130222592 | Gieseke | Aug 2013 | A1 |
20130342333 | Hutchings | Dec 2013 | A1 |
20140218529 | Mahmoud | Aug 2014 | A1 |
20140222323 | Purushothaman et al. | Aug 2014 | A1 |
20140375476 | Johnson | Dec 2014 | A1 |
20150124096 | Koravadi | May 2015 | A1 |
20150158499 | Koravadi | Jun 2015 | A1 |
20150251599 | Koravadi | Sep 2015 | A1 |
20150352953 | Koravadi | Dec 2015 | A1 |
20160019497 | Carvajal | Jan 2016 | A1 |
20160036917 | Koravadi et al. | Feb 2016 | A1 |
20160048966 | Kuehnle | Feb 2016 | A1 |
20160210853 | Koravadi | Jul 2016 | A1 |
20160381571 | Koravadi et al. | Dec 2016 | A1 |
20170011633 | Boegel | Jan 2017 | A1 |
20170222311 | Hess | Aug 2017 | A1 |
20170254873 | Koravadi et al. | Sep 2017 | A1 |
20170276788 | Wodrich | Sep 2017 | A1 |
20170315231 | Wodrich | Nov 2017 | A1 |
Number | Date | Country |
---|---|---|
WO2011090484 | Jul 2011 | WO |
Number | Date | Country | |
---|---|---|---|
20170158133 A1 | Jun 2017 | US |
Number | Date | Country | |
---|---|---|---|
62263187 | Dec 2015 | US |