Patent Grant
10845809
The present invention relates to an automatic driving device that automatically controls a vehicle to a destination by automatic steering and automatic speed control.
Various technologies for recognizing objects (vehicles, pedestrians, structures, and the like) around a host vehicle and road markings/signs (pavement paints such as lane-dividing lines, signs such as stop sign, and the like) using external recognition sensors such as in-vehicle cameras and radars have been proposed. Various technologies also have been proposed to control a host vehicle using these technologies and improve the security and comfort of occupants; additionally, automatic driving technologies for traveling automatically to a destination by controlling the steering and speed of the host vehicle in an automated manner have started to be proposed. In order to achieve such an automatic driving technology, it is required to precisely judge the situation even under a complicated environment to control the vehicle and, for example, it is necessary to be able to deal with even situations without precedent by applying artificial intelligence and machine learning to this situation judgment.
There is a technology for controlling a vehicle in such a manner that information necessary for predicting travel control from host vehicle position information and vehicle information is stored in a learning database as history information and, on the basis of, for example, road information ahead of the host vehicle position and a past traveling pattern predicted from the history information, future travel control of the host vehicle is predicted, while control command values for a transmission, regenerative brake ON/OFF, traveling mode, and the like are created on the basis of travel control predicted by a learning unit (refer to PTL 1).
PTL 1: JP 2013-237290 A
With the technology disclosed in PTL 1, however, since the vehicle is controlled on the basis of the travel control predicted by the learning unit, the vehicle is controlled as predicted even when the prediction fails.
Nevertheless, the vehicle control applied in PTL 1 is restricted to a transmission, regenerative brake ON/OFF, traveling mode, and the like and safety is not impaired even when the prediction by the learning unit fails. In this manner, machine learning and artificial intelligence have a possibility of failing predictions or making unexpected judgment depending on learning methods and experience values (accumulated amount of databases, and the like) and thus there has been a problem such as difficulties in applying machine learning and artificial intelligence to vehicle control or the like concerning safety.
The present invention has been made in view of the above circumstances and it is an object of the present invention to provide an automatic driving device using intelligence such as machine learning and artificial intelligence for achieving safety and natural automatic driving.
In order to solve the above problem, the present invention provides an automatic driving device including a control program for inputting outside information and vehicle information, and outputting a target control value for a vehicle, in which the control program has a first program for generating a first target control amount on the basis of a dynamically changing algorithm, and a second program for generating a second target control amount on the basis of a prescribed algorithm.
According to the present invention, automatic driving under a complicated environment and vehicle control reflecting personal preferences can be achieved by the dynamically changing algorithm such as machine learning or artificial intelligence, and at the same time vehicle control ensuring the operation is fulfilled by the prescribed algorithm according to traffic rules or with upper and lower limit values of control amount fixed in advance, whereby the quality as an automatic driving device can be secured. Specifically, safety can be secured by the latter prescribed algorithm ensuring the operation.
Further features relating to the present invention will become apparent from the description in this specification and the accompanying drawings. In addition, problems, configurations, and effects other than those mentioned above will be clarified by the description of the following embodiments.
Working examples will be described below with reference to the drawings.
The first program 102 is constituted by a dynamically changing algorithm such as learning function or artificial intelligence and is a program highly likely to output different values for the same input with the lapse of time. The dynamically changing algorithm is a processing technique for generating a target control amount on the basis of artificial intelligence or learning function. The learning function here is a function to store input information, executed maneuvers and the like and to conduct optimization so as to perform a more appropriate process grounded on such storage and, for example, is capable of reflecting tastes of a driver on acceleration/deceleration, lateral acceleration, inter-vehicle distance, and the like in automatic driving. The learning function also includes ones that use machine learning and artificial intelligence to understand input information and make appropriate output.
The second program 103 is constituted by a prescribed algorithm based on rules and regulations defined in advance (hereinafter referred to as “rule base”) and is a program that outputs the same value or a range thereof for the same input. The prescribed algorithm is a processing technique for generating a target control amount on the basis of at least traffic rules and driving morals. However, this does not apply when rules and regulations are changed. When the prescribed algorithm is applied to automobiles, rules and regulations correspond to traffic rules and driving morals, but it is needless to say that these rules and regulations differ depending on the country and region to which they apply.
As described above, by separating a program having a dynamically changing algorithm and a program having a prescribed algorithm as a constitution of a program, the effect of reducing the number of person-hours for carrying out the quality check of programs, such as maintenance and updating, is expected.
A trajectory/speed planning unit (learning function) 102a in
A trajectory/speed planning unit (rule base) 103a in
A trajectory/speed planning unit (artificial intelligence) 102b in
The trajectory/speed planning unit (rule base) 103b in
The outside environment recognition device 401 acquires information regarding surrounding environment of the host vehicle and, for example, is an in-vehicle stereo camera for photographing the front of the host vehicle, or four in-vehicle cameras for separately photographing the surrounding environment on the front, rear, right side, and left side of the host vehicle. These in-vehicle cameras use the obtained image data to detect the shape and position of an object such as a stationary three-dimensional object, a moving object, a pavement paint such as a lane marking, and a sign around the host vehicle, and additionally has a function of determining whether a road surface is acceptable for the host vehicle to travel on by detecting irregularities or the like on the road surface. The stationary three-dimensional object is, for example, a parked vehicle, a wall, a pole, a pylon, a curbstone, or a car stop. The moving object is, for example, a pedestrian, a bicycle, a motorcycle, or a vehicle. Hereinafter, both of the stationary three-dimensional object and the moving object are collectively referred to as an obstacle.
The shape and position of the object are detected using a pattern matching technique or other known technologies. The position of the object is expressed, for example, using a coordinate system having the origin at the position of the in-vehicle camera that photographs the front of the host vehicle. Then, the obtained information such as the type, distance, and direction of the object is output to the automatic driving device 400 using a leased line, a CAN or the like.
Note that an approach to output the image obtained by the in-vehicle camera to the automatic driving device 400 using a leased line or the like and process the image data in the automatic driving device 400 may be employed. In addition to the in-vehicle cameras, it is also possible to use a radar that measures the distance to an object using millimeter waves or a laser, a sonar that measures the distance to an object using ultrasonic waves, and the like such that the obtained information such as the distance to the object and the direction thereof is output to the automatic driving device 400 using a leased line, a CAN, or the like. Furthermore, a communication device for communicating with the outside of the host vehicle may be included in the outside environment recognition device 401, so as to communicate with a vehicle around the host vehicle to exchange information on position and speed, and the like, or communicate with an communication unit on roadside to exchange information that cannot be detected by a sensor mounted on the host vehicle (such as information on an obstacle in a blind spot of the host vehicle).
The steering device 403 is constituted by an electric power steering, a hydraulic power steering, and the like capable of controlling the steering angle with an electric or hydraulic actuator or the like according to an activation command from the outside. The steering device 403 can control the trajectory of the host vehicle to the target trajectory by activation of the actuator.
The activating device 404 is constituted by an engine system capable of controlling the engine torque with an electric throttle or the like according to an activation command from the outside, or an electric power train system capable of controlling the activation force with a motor or the like according to an activation command from the outside, or the like.
The braking device 405 is constituted by an electric brake, a hydraulic brake, or the like capable of controlling the braking force with an electric or hydraulic actuator or the like according to a braking command from the outside. The braking device 405 can control the speed of the host vehicle to the target speed by activation of the actuator.
The sound producing device 406 is constituted by a speaker or the like and is used for outputting warnings, voice guidance and the like to the driver.
The display device 407 is constituted by a display of a navigation device or the like, a meter panel, a warning lamp, and the like. In addition to a maneuver screen of the automatic driving device 400, the display device 407 displays, for example, a screen on which the traveling state of the host vehicle can be visually expressed.
The automatic driving button 402 is a maneuvering member provided at a position maneuverable by the driver and outputs a signal to switch the operation of the automatic driving device 400 to the automatic driving device 400 on the basis of the maneuver by the driver. Note that the automatic driving button 402 can be installed as a switch at a place easily maneuverable by the driver around the steering wheel, and additionally, in a case where the display device 407 is a touch panel type display, a button may be displayed on the display device 407 so as to be maneuverable by the driver.
Next, processing procedures of the automatic driving device 400 will be described with reference to flowcharts.
In process S501 in
In process S502, the road information is acquired and the procedure proceeds to process S503. Here, the road information is map data around the current host vehicle and this map data includes shape data close to the actual road shape expressed by polygons, polylines, and the like, traffic restriction information (speed limit, permitted vehicle type, and the like), lane classification (main line, overtaking lane, climbing lane, through lane, left turning lane, right turning lane, and the like), whether traffic lights and signs are present (if any is present, position information thereon), and the like.
In process S503, a process of grasping the traveling environment around the host vehicle is carried out using the outside information acquired in process S501 and the road information acquired in process S502, and the procedure proceeds to process S504. Specifically, the outside information on an obstacle or the like is placed on the map data and the lane position where the host vehicle can travel, the available turning space in the intersection, and the like are detected.
In process S504, it is judged whether the automatic driving is in progress. When the automatic driving is in progress, an automatic driving process in S505 is executed to terminate the series of processes and the procedure returns to process S501. On the other hand, when the automatic driving is not in progress in process S504, a learning process in S506 is executed to terminate the series of processes and the procedure returns to process S501.
In process S601 in
In process S602, it is determined whether the host vehicle has traveled on a learning object scene, on the basis of the information stored in process S601. When it is determined that the host vehicle has traveled on a learning object scene, the procedure proceeds to process S603 and, when it is determined that the host vehicle has not traveled on a learning object scene, the series of processes is terminated. Here, a learning object scene is, for example, as follows.
In process S603, it is determined whether the manner of the host vehicle in the learning object scene determined in process S602 was traveling in compliance with the rule base. When the manner of the host vehicle was traveling in compliance with the rule base, the procedure proceeds to process S604 and, when the manner of the host vehicle was not traveling in compliance with the rule base, the series of processes is terminated. Here, the traveling in compliance with the rule base means traveling that abides by traffic rules and is not contrary to driving morals.
In process S604, the surrounding environment and the traveling pattern of the host vehicle are learned in association with each other and the series of processes is terminated. Here, for example, specific learning contents are as follows.
In process S701 in
In process S702, the target control amount is generated by the second program and the procedure proceeds to process S703. In this example, the target trajectory and the target speed of the host vehicle are generated according to a trajectory/speed plan based on the rule base.
In process S703, a target control value which is the final target control amount is generated from the first target control amount generated in process S701 and the second target control amount generated in process S702 and the procedure proceeds to process S704.
In process S704, a control parameter for causing the host vehicle to travel according to the target trajectory and the target speed of the host vehicle, which are the target control value generated in process S703, is calculated and the procedure proceeds to process S705. Here, examples of the control parameter include a target steering torque, a target engine torque, and a target brake pressure.
In process S705, each of the control parameters calculated in process S704 is output to the steering device 403, the activating device 404, and the braking device 405 and the series of processes is terminated.
Note that the target steering torque for implementing the target steering angle is cited as the control parameter to be output to the steering device 403, but it is also possible to directly output the target steering angle, depending on the configuration of the steering device 403. In addition, the target engine torque, the target brake pressure, and the like for implementing the target speed are cited as the control parameters to be output to the activating device 404 and the braking device 405, but it is also possible to directly output the target speed, depending on the configuration of the activating device 404 and the braking device 405.
In process S801 in
Next, an example of the learning process described in
When the host vehicle 900 reaches a point A while traveling at a speed V1, the host vehicle 900 begins to decelerate toward a stop line 902 where a stop sign 901 is present, and temporarily stops at a point B on the near side of a point S of the stop line 902. Thereafter, the driver of the host vehicle 900 confirms safety in the intersection and, when judging that safety is assured, restarts the host vehicle 900 to pass through a point C. First, this series of traveling scenes is determined to be learning object scenes by the determination in process S602 in
The contents learned here are a speed pattern 903 (more specifically, the average deceleration between the points A and B and the average acceleration between the points B and C), the stop position (the distance between B and S), the start timing (the stopping time at the point B), and the like of the host vehicle 900.
When the host vehicle 910 reaches a point B1, the driver of the host vehicle 910 judges that there is a sufficient inter-vehicle distance between the vehicle 911 at a point A1 and the vehicle 912 at a point C1 and turns on the blinker to change the lane with a locus indicated by a dotted line 913. Then, at the time point when the host vehicle 910 reaches a point B2, the lane change is completed and the blinker is turned off. Note that it is assumed that the vehicle 911 is located at a point A2 and the vehicle 912 is located at a point C2 at the time when the lane change is completed and sufficient inter-vehicle distances are kept between the respective vehicles. First, this series of traveling scenes is determined to be learning object scenes by the determination in process S602 in
The contents learned here are the traveling locus 913, the lateral acceleration, the inter-vehicle distances to surrounding vehicles (the distance between A1 and B1 and the distance between B1 and C1 at the beginning of lane change), and the like of the host vehicle 910.
When the host vehicle 1000 reaches a point A while traveling at a speed V1, the host vehicle 1000 begins to decelerate toward a stop line 902 where a stop sign 901 is present, and cancels the deceleration in the vicinity of a point S of the stop line 902 to enter the intersection without stopping. Thereafter, the driver of the host vehicle 1000 confirms safety in the intersection and, when judging that safety is assured, accelerates the host vehicle 1000 to pass through a point C. First, this series of traveling scenes is determined to be learning object scenes by the determination in process S602 in
Therefore, the traveling behavior at this time is not learned.
When the host vehicle 1010 reaches a point B1, the driver of the host vehicle 1010 turns on the blinker toward a space between the vehicle 1011 at a point A1 and the vehicle 1012 at a point C1 to change the lane. Then, at the time point when the host vehicle 1010 reaches a point B2, the lane change is completed and the blinker is turned off. Note that it is assumed that the vehicle 1011 is located at a point A2 and the vehicle 1012 is located at a point C2 at the time when the lane change is completed and the vehicle 1011 behind has extended the inter-vehicle distance for safety. First, this series of traveling scenes is determined to be learning object scenes by the determination in process S602 in
As described so far, by carrying out the learning process only when the traveling complies with the rule base, appropriate learning is enabled.
Next, an example of the target control value generation process described in
When the traffic light at the intersection ahead of the host vehicle 1100 changes from green to yellow at the time of reaching a point A, the speed pattern of the host vehicle 1100 is calculated and the target speed is calculated on the basis of the calculated speed pattern. First, when a deceleration pattern based on the first program is calculated as a dotted line 1110, the host vehicle 1100 cannot pull up on the near side of a stop line 1102 and accordingly the target speed is output as a value for maintaining the speed to pass through the intersection as indicated by a dotted line 1112. Next, when a deceleration pattern based on the second program is calculated as a broken line 1111, it is judged that the host vehicle 1100 can pull up on the near side of the stop line 1102 and the target speed is output as indicated by a broken line 1113. The target speed based on the second program is an output in compliance with the rule base, which abides by the traffic rules (trying to stop when the traffic light turns yellow) and is not contrary to driving morals (stopping to an extent that does not result in a sudden braking), and this output has obtained an output of the target speed that can cause the vehicle to stop at the stop line. Therefore, the target speed based on the first program is judged to deviate from the target speed based on the second program and the target speed based on the second program is adopted.
However, the above-mentioned driving morals may differ depending on the countries and regions to which they apply, such that the trajectory/speed plan based on the second program is not always adopted.
As described so far, when the target control value based on the first program is compared with the target control value based on the second program and the target control value based on the first program deviates from the target control value based on the second program, the target control value based on the second program is adopted. Therefore, an output in compliance with the rule base is finally selected.
Next, a second embodiment of the present invention will be described with reference to
The second embodiment of the present invention is obtained by replacing the target control value generation process illustrated in
In process S1301 in
In process S1302, it is determined whether the second target control amount has been adopted in process S1301. When it is determined that the second target control amount has been adopted, the procedure proceeds to process S1303 and the second target control amount is set as the target control value in process S1303; then, the series of processes is terminated. On the other hand, when it is not determined in process S1302 that the second target control amount has been adopted, the procedure proceeds to process S1304 and the first target control amount is set as he target control value in process S1304; then, the series of processes is terminated.
As described so far, by using the method of selecting the first target control amount and the second target control amount depending on the environment around the host vehicle, traveling in compliance with the rule base is enabled as a result.
Next, a third embodiment of the present invention will be described with reference to
The third embodiment of the present invention is obtained by replacing the automatic driving process illustrated in
In process S1501 in
In process S1502, it is determined whether the first target control amount is within a prescribed range. When it is determined that the first target control amount is within the prescribed range, the procedure proceeds to process S1503 and the first target control amount is set as the target control value in process S1503; then, the procedure proceeds to process S1505. On the other hand, when it is determined in process S1502 that the first target control amount is not within the prescribed range, the procedure proceeds to process S1504. The first target control amount is corrected to fall within the prescribed range and set as the target control value in process S1504; then, the procedure proceeds to process S1505.
Since processes S1505 and 51506 are the same as processes S704 and S705 described in
Similarly to
When the traffic light at the intersection ahead of the host vehicle 1600 changes from green to yellow at the time of reaching a point A, the speed pattern of the host vehicle 1600 is calculated and the target speed is calculated on the basis of the calculated speed pattern. First, when a deceleration pattern based on the first program is calculated as a dotted line 1610, the host vehicle 1600 cannot pull up on the near side of a stop line 1602 and accordingly the target speed is output as a value for maintaining the speed to pass through the intersection as indicated by a dotted line 1612. Next, when the prescribed range based on traffic rules and driving morals is computed, the range of a region 1611 is output and accordingly it is judged that the host vehicle 1600 can pull up on the near side of the stop line 1602 by selecting this range. Then, when the target speed is corrected within the range of the region 1611, the corrected target speed is given as indicated by a broken line 1613 and this target speed is adopted.
As described so far, also using the method of correcting the first control amount to the prescribed range, traveling in compliance with the rule base is finally enabled.
Next, a fourth embodiment of the present invention will be described with reference to
The fourth embodiment of the present invention has a configuration obtained by changing the function equivalent to the first program in the configuration illustrated in
A driving task designation unit (learning function) 102c in
A driving task designation unit (artificial intelligence) 102d in
The driving task here represents the driving state of the host vehicle, such as preceding vehicle following, lane tracking, lane change, going straight ahead/right turn/left turn at intersection, and parking, and is a control state serving as a guide for movement for trajectory/speed planning units in
In process S1901 in
In process S1902, a target control value is generated on the basis of the second program and the procedure proceeds to process S1903.
Since processes S1903 and 51904 are the same as processes S704 and S705 described in
As explained so far, the movement (driving task) of the host vehicle at the manner level is designated by the first program and the trajectory/speed plan as the target control value is generated by the second program based on the rule base, whereby traveling in compliance with the rule base is enabled.
Note that the above description is merely an example and, in interpreting the invention, there is no limitation or restriction on the correspondence relationship between the description items of the above embodiments and the description items of the claims. For example, in the above-described embodiments, a passenger car is supposed as the host vehicle, but the present invention is also applicable to automatic traveling of a construction machine, a robot, or the like.
While the embodiments of the present invention have been described so far in detail, the present invention is not limited to the above-described embodiments and various design changes can be made without departing from the spirit of the present invention described in the claims. For example, the aforementioned embodiments have been described in detail in order to make the description of the invention easy to understand. Therefore, the embodiments are not necessarily limited to the ones provided with the whole configurations that have been described. In addition, part of the configuration of a certain embodiment can be replaced with the configuration of another embodiment, while it is also possible to add the configuration of a certain embodiment to the configuration of another embodiment. Part of the configuration of each of the embodiments can be subjected to addition, deletion, and replacement of another configuration.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2016-155726 | Aug 2016 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2017/026028 | 7/19/2017 | WO | 00 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2018/030082 | 2/15/2018 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 20060095195 | Nishimura et al. | May 2006 | A1 |
| 20080162027 | Murphy et al. | Jul 2008 | A1 |
| 20140371981 | Nordbruch et al. | Dec 2014 | A1 |
| 20150057905 | Niwa et al. | Feb 2015 | A1 |
| 20150158486 | Healey et al. | Jun 2015 | A1 |
| 20150308837 | Cudak | Oct 2015 | A1 |
| 20160132705 | Kovarik et al. | May 2016 | A1 |
| 20170169208 | Jantz | Jun 2017 | A1 |
| 20170285585 | Weast | Oct 2017 | A1 |
| 20180089563 | Redding | Mar 2018 | A1 |
| Number | Date | Country |
|---|---|---|
| 6-206477 | Jul 1994 | JP |
| 8-44999 | Feb 1996 | JP |
| 2006-131055 | May 2006 | JP |
| 2009-205646 | Sep 2009 | JP |
| 2013-237290 | Nov 2013 | JP |
| 2015-722 | Jan 2015 | JP |
| 2015-41241 | Mar 2015 | JP |
| Entry |
|---|
| International Search Report (PCT/ISA/210) issued in PCT Application No. PCT/JP2017/026028 dated Oct. 17, 2017 with English translation (four (4) pages). |
| Japanese-language Written Opinion (PCT/ISA/237) issued in PCT Application No. PCT/JP2017/026028 dated Oct. 17, 2017 (four (4) pages). |
| Extended European Search Report issued in European Application No. 17839155.3 dated Feb. 26, 2020 (eight (8) pages). |
| Number | Date | Country | |
|---|---|---|---|
| 20190317499 A1 | Oct 2019 | US |
