Patent Grant
12139070
This application claims priority to European Patent Application Number 20210810.6, filed Nov. 30, 2020, the disclosure of which is hereby incorporated by reference in its entirety herein.
Nowadays, the usage of motorized vehicles is in constant expansion. In urban areas, more and more road users are mixing together on the roads (e.g., vehicles such as cars, trucks or pedestrians and other users). The increase in the number of road users leads to an increase in the risk of accidents between them. Drivers needs to be increasingly attentive, and vehicles equipped with Advanced Driver-Assistance Systems (ADAS) can help limit and anticipate hazardous or dangerous situations.
One of the risks in the event of an hazardous situation is the fact that the driver is not aware of upcoming or forthcoming action on the vehicle by driver-assistance systems (e.g., braking, steering, horning) to avoid the danger. This can lead to unexpected reaction of the driver, since he is still responsible of the vehicle's movement, over the driver-assistance systems.
This risk is amplified by the fact that there are blind spot areas (or zones) due to the conventional construction of the vehicle. In addition to the two known blind spot areas at the rear of the vehicle (car), there are two others blind spot areas for the driver in his line of sight or field of view. These are due to the two front car pillars. Since the two front pillar blind areas mainly depend on where the driver look is pointing (or driver's gaze or field of view is directed), we can define them as dynamic blind areas. As soon as a road user is inside one of the blind spot areas, the driver cannot see him as it is not inside his field of view. As a result, the driver is not informed of his presence, which increase the risk of an accident.
With the evolution of driver-assistance systems, the risks of accident are reduced but still existing. These systems are composed of radars, cameras or lidars (frontal and lateral) which allow for a 360 degree identification of road users. However, driver-assistance systems require an attention of the driver in certain specific scenarios.
The present disclosure improves the situation over the prior art. In particular, a purpose of the present disclosure is to give to the driver the sense of responsibility, and to help him during driving, while giving the relevant level of information without transferring the whole responsibility to the driver-assistance system.
The present disclosure relates to the field of safe driving of vehicles. Disclosed methods and systems can be used for example in a vehicle, such as a car. In aspects, described is a method for driver assistance to be implemented in a driver-assistance system of a vehicle, including the following steps (operations): performing an environment analysis around the vehicle, identifying a potential risk around the vehicle based on the performed environment analysis, identifying a field of view of a driver, and determining, dynamically when the vehicle circulates, whether or not the identified potential risk is located in the field of view of the driver and is hidden in a blind spot area of the vehicle, and when the identified potential risk is located in the field of view of the driver and the identified potential risk is hidden in the blind spot area of the vehicle, informing the driver to change a driver's position in order to be able to see the identified potential risk. This informs the driver that it is his responsibility to change its position in order to acknowledge himself the danger. The change in his position could be either changing the head position, changing his field of view, moving his body, or the like. The improvement of the safety may be done with the increase of the responsibility of the driver, without always giving him alerts or unnecessary information, and without giving him the impression that the driver-assistance system is driving alone, without driver supervision. This anticipates the risks (or targets) and dangers, and increases the acceptability of information by the driver.
The steps of performing the environment analysis around the vehicle, identifying the potential risk or target around the vehicle based on the performed environment analysis, and identifying the field of view of the driver, that is to say based on the data retrieved by the sensors, are well known by the one skilled in the art and are conventional. Non-limitative examples of potential risk or targets are pedestrian, cyclist, vehicle, animals, trucks, train and the like.
Advantageously, the method further includes the following steps: determining whether or not the driver has changed the driver's position in order to be able to see the identified potential risk, and when the driver has changed the driver's position in order to be able to see the identified potential risk, repeating the steps. The steps may be repeated, as necessary. to allow a continuous safe driving. It is to be noted, that all the steps of the present disclosure may be repeated, provided it is not in contradiction with other steps.
Advantageously, the method further includes the following steps: when the driver has not changed the driver's position in order to be able to see the identified potential risk, informing the driver to immediately change the driver's position in order to be able to see the identified potential risk. This further increases the information level of the driver, so that he still considers himself as responsible of the driving. This shows him that the driving remains its responsibility.
Advantageously, the method further includes the following steps: determining whether or not a critical point of the identified potential risk will be reached within a double check period, when the critical point of the identified potential risk will be reached within the double check period, informing the driver of a forthcoming takeover of the driver-assistance system and then allowing the forthcoming takeover of the driver-assistance system. This informs the driver of the forthcoming action of the driver-assistance system, such as braking, steering, changing direction, drifting, horning, blinking or any other driver-assistance action.
Advantageously, the method further includes the following steps: determining whether or not a critical point of the identified potential risk will be reached within a general safety period, when the critical point of the identified potential risk will be reached within a general safety period, informing the driver of a forthcoming takeover of the driver-assistance system, and then allowing the forthcoming takeover of the driver-assistance system. This may permit for a general safety period to take over the control of the vehicle in all cases, so that the safety is improved without further checking, and may permit for the minimal general safety level to be reached.
Advantageously, the method further includes the following step: when the identified potential risk is not located in the field of view of the driver and the identified potential risk is not masked by the blind spot of the vehicle, informing the driver that the identified potential risk is without immediate danger. This gives background information about danger to the driver.
Advantageously, the step: informing the driver to change the driver's position in order to be able to see the identified potential risk, is performed by an orange continuous warning in a human machine interface of the vehicle.
Advantageously, the step: informing the driver to immediately change the driver's position in order to be able to see the identified potential risk, is performed by an orange blinking warning and/or a repeated sound in a human machine interface of the vehicle.
Advantageously, the step: informing the driver of a forthcoming takeover of the driver-assistance system is performed by red continuous warning and/or a continuous alert sound in a human interface of the vehicle.
Advantageously, the human machine interface of the vehicle is a display unit for visual information, including a sound emitting device for acoustic information, in order to inform the driver. In particular, the display unit can emit or generate texts, color, pictogram, light, and any combination thereof. This informs the driver with the relevant level of information, while increasing its responsibility, not giving over information disturbing him, not giving him the impression that the driver-assistance system will do the driving instead of the driver. In particular, reference is made to “Human Factors for Connected Vehicles: Effective Warning Interface Research Findings” published in September 2014 by the National Highway Traffic Safety Administration in the USA, Report No. DOT HS 812 068, which is incorporated by reference. This document explains how the relevant level of information is important for the driver.
Advantageously, the step: determining whether or not the identified potential risk is located in the field of view of the driver and is hidden in a blind spot area of the vehicle, is performed dynamically when the vehicle circulates or is moving. This increases the safety while driving, and not only when the vehicle is parked or stopped (e.g., at a traffic light).
The present disclosure also relates to a driver-assistance system having a processor arranged to implement the method for driver assistance according to the present disclosure. In other words, the present disclosure also relates to a data processing driver-assistance system comprising a processor configured to perform the steps of the method for driver assistance. This provides a driver-assistance system with an improved level of safety, while not disturbing the driver with not necessary volume of information and confirming him that the driving responsibility remains with him.
The present disclosure also relates to a vehicle integrating the driver-assistance system according to the present disclosure.
Advantageously, the vehicle further includes at least one external sensor, at least one internal sensor and a human machine interface. This provides a vehicle with an improved level of safety, while not disturbing the driver with not necessary volume of information and confirming him that the driving responsibility remains with him.
The present disclosure also relates to a computer program comprising instructions which, when the program is executed by a processor, cause the processor to control the execution of the steps of the method according to the present disclosure. In other words, the present disclosure also relates to a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method according to the present disclosure. This provides a computer program increasing the with an improved level of safety, while not disturbing the driver with not necessary volume of information and confirming him that the driving responsibility remains with him.
Other features, purposes and advantages of the disclosure will become more explicit by means of reading the detailed statement of the non-restrictive embodiments made with reference to the accompanying drawings.
The vehicle 100 has a left pillar 13 and a right pillar 14. The vehicle 100 has also a left rear-view mirror 11 and a right rear-view mirror 12. When a driver 101 is present in the vehicle 100 in order to drive the vehicle 100, the vehicle presents some blind spot areas 1, 2, 3, 4, wherein the driver 101 cannot see correctly (or cannot see at all) the objects.
To be more specific: there is a left front blind spot area 1 due to the left front pillar 13, there is a right front blind spot area 2 due to the right front pillar 14, there is a left rear blind spot area 3 due to the fact that the left rear-view mirror 11 cannot show all the left rear side, and there is a right rear blind spot area 4 due to the fact that the right rear-view mirror 12 cannot show all the right rear side. There is thereby a need to inform or alert the driver of the presence of any dangerous object in the blind spot areas.
The vehicle 100 further includes a set of external sensors 16 (or a set of outside sensors 16), such as video cameras, radars, lidars arranged to perform an environmental analysis 200 around the vehicle 100. The driver-assistance system has memory means, such as RAM and ROM, along with identification means, such as a neural network, in order to identify a potential risk around the vehicle 100 based on the performed environment analysis. The driver-assistance system is able to detect and to recognize an obstacle around the vehicle 100.
The vehicle 100 further includes a set of internal sensors 15 (or a set of inside sensors 15), such as an internal camera (mono-focal or multi-focal) or an eye-tracking device, in order to identify a field of view 102 of the driver 101. This could also relate to the identification of a line of sight of the driver, the eye positions or a position or an orientation of the head of the driver 101, and the position of the driver 101 in its seat.
The line of sight of the driver may be checked by the direction of the face and the eye gaze. In particular, the field of view 102 of the driver may be tracked to define where the driver 101 looks. Some example of regions of interest or regions of distraction are: the radio equipment, the steering wheel, the internal rear-view mirror, the windscreens, the lateral windows, the left rear-view mirror 11, the right rear-view mirror 12, the passenger seats.
The vehicle 100 further includes a human machine interface 17, such as a display unit and a sound emitting device, in order to inform the driver 101.
Identification of a potential risk 700 is performed and is labelled 201, preferably at 360 degrees around the vehicle 100. This permits for the detection of the potential risk 700, as illustrated with more details in the driving scenario of
The step of determining whether or not (i.e., checking) the identified potential risk 700 is located in the field of view 102 of the driver 101 and is hidden in a blind spot area 1, 2, 3, 4 of the vehicle, labelled 203, is performed.
If the response to the above determination 203 is YES, an information is sent to the driver 101 to change the driver's position in order to be able to see the identified potential risk 700. This information may include a LEVEL 1A warning 301 (or medium-level warning) which is emitted, for example an orange continuous warning on the display unit.
If the response to the above determination 203 is NO, an information is sent to the driver 101 that a risk is present without immediate danger 206. This information may include a LEVEL 1B warning 302 (or low-level warning, which may also to set as a medium-level warning) which is emitted, for example an orange blinking warning on the display unit and a low-level sound or low-speed sound (e.g., one beep per second).
Then, the step of determining whether or not (i.e., checking) the driver 101 has changed the driver's position in order to be able to see the identified potential risk 700, labelled 205, is performed.
If the response to the above determination 205 is YES, the step labelled 202, of analyzing and identifying the field of view, is repeated. In other words, it means that the driver 101 correctly changed its position in the vehicle or in its seat or correctly changed its field of view, so as to correctly acknowledge the potential risk 700.
If the response to the above determination 205 is NO, the step of informing the driver 101 to immediately change the driver's position in order to be able to see the identified potential risk 700, is performed. This information may include a LEVEL 1C warning 305 (or medium-level warning) which is emitted, for example an orange blinking type warning on the display unit and a medium-level sound or medium-speed sound (e.g., two beeps per second).
Then, the step of determining whether or not (i.e., checking) a critical point of the identified potential risk 700 will be reached within predetermined period, labelled 207, is performed.
If the critical point of the identified potential risk 700 will be reached within the predetermined period, the step of informing the driver 101 of a forthcoming takeover of the driver-assistance system and then allowing the forthcoming takeover of the driver-assistance system, labelled 208, is performed. This information may include a LEVEL 2 warning 304 (or high-level warning) which is emitted, for example a red continuous warning on the display unit and a high-level sound or continuous sound 305.
The predetermined period can be set for example at 3 seconds or less. It is to be noted that the predetermined period can be set differently if the driver has changed its position at the above-mentioned step 205. Indeed, in this case, the predetermined period is set as a double check period, which is preferably lower than the predetermined period of the labelled step 206, wherein the risk was without immediate danger. The goal is to encourage the driver to be responsible for his driving. The double check period can be set at 2 seconds or less.
Further, this permits two modes to be distinguished between: the first mode 350, wherein the driver 101 is looking in direction of the potential risk 700, so that the attention assistance is performed, and the second mode 351, in contrast, wherein the driver-assistance feature will take over the control of the vehicle.
If the risk without immediate danger defined at the step labelled 206 is still present after a short period of time (e.g., few seconds), an information is sent to the driver 101. This information may include a LEVEL 1D warning 303 (or medium-level warning) which is emitted, for example an orange blinking type warning on the display unit and a medium-level sound or medium-speed sound (e.g., two beeps per second). It is to be noted that LEVEL 1A, 1B, 1C and 1D are all a WARNING LEVEL 1.
In all cases, if the critical point of the identified potential risk 700 will be reached within a general safety period (e.g., 0.5 seconds), the step of informing the driver 101 of a forthcoming takeover of the driver-assistance system and then allow the forthcoming takeover of the driver-assistance system, labelled 208, is performed. The time to collision may be different between the two modes 350, 351, but the general safety period is set to be safe. That is to say, the initial predetermined period can be set with a higher value than the double check period, and the double check period can be set with an higher value than the general safety period. Further, this set of values can be changed according to the driving mode (safe, comfort, sport, baby on board, city driving wherein the speed is limited to 50 or 60 km/h, motorway driving wherein the speed is usually between 80 and 140 km/h, rural driving wherein the speed is between 60 and 80 km/h, etc.).
In other words, a time to collision is monitored and the steps are looped continuously. If the time to collision (for any reason, like target/vehicle direction or speed change) is increasing until above a specific time to collision limit (above which there is no risk of collision) while the steps are looped, there is no need to allow the forthcoming takeover of the driver-assistance system. The time to collision is adjusted in consequence and continuously when the vehicle circulates (for any reason, like target/vehicle direction or speed change). In other words, it means that if the driver 101 correctly changed its position in the vehicle or in its seat or correctly changed its field of view, so as to correctly acknowledge the potential risk 700 and took action to avoid it, or if there is no risk anymore due to time to collision above the specific time to collision (target change), there is no need any more to alert him of the forthcoming takeover of the driver-assistance system and then allowing the forthcoming takeover of the driver-assistance system, as the risk has been safely identified and avoided.
This assistance should check when a risk is estimated to be critical rather soon (e.g., Time To Collision calculated) and trigger a warning as soon as a defined threshold is reached. For example a situation where the time to collision is below three second, the system may trigger a first WARNING LEVEL 1 (ex: visual) and hold it until the critical point is reached (that is to say when the driver-assistance will take action on the vehicle motion), WARNING LEVEL 2 (ex: visual+sound). Before this critical point is reached, it can be considered that the collision can be avoided by a driver manipulation. Thus, he should be aware of the danger approaching. In a case where the driver has not seen the danger because of blind spot area inside the car, a specific warning DANGER IN BLIND SPOT (ex: haptic+sound in the pillar, different from LEVEL 1 and 2 HMI and catching driver attention in direction of the risk) may be triggered. This should catch the driver attention and have his mind in the loop of the vehicle driving. This will help him anticipate the risk and the associated WARNING LEVEL 2 while the driver-assistance system feature will take action.
For this, calculation may be performed to measure the position of the driver's eyes and determine whether his field of view (or line of sight) is masked by one of the pillar for instance by measuring the eye gaze. This will allow to determine the blind zone dynamically.
For the warning logic to catch driver attention, reference is made to the document DOT HS 812 068 from National Highway Traffic Safety Administration “Human Factors for Connected Vehicles: Effective Warning Interface Research Findings.” which is incorporated by reference.
This document summarizes studies that shows effectiveness of warning drivers of hazardous situations. Some factors out of those studies will be used in this invention to have effective alert scheme, namely visual (color and fix/blinking) and sound (sequential/continuous). Those factors combined in the alert scheme allow effective alert for the different stage of the invention. Effectiveness is defined by a combination of importance, urgency, annoyance, and acceptability that the participants of study rated in different situation experienced.
The driver-assistance system reaction is defined by the steps described in the present disclosure.
A potential risk can be characterized by a target (that is to say a road user identified by the conventional driver-assistance system) that presents a risk of collision with the vehicle. The driver-assistance system permits characterization of the risk and predicts his trajectory.
Using all those data, an algorithm evaluates if the driver is looking in the direction of the risk and if the risk is hidden by a blind spot area. This can be a look direct into the risk or through the external mirrors when the risk is in the rear field of view.
The
The display unit is organized with different zones, 401-406, with a left display zone 401, 403, 405 and a right display zone 402, 404, 406. When a low-level danger is detected, the display zones 405 and 406 are highlighted in green and may be highlighted according to the side (left or right) of the risk (left side on the upper schematic representation of the display unit of
It is to be noted that the step of informing the driver 101 to change the driver's position in order to be able to see the identified potential risk can be adapted according to the driving mode mentioned above. In particular, the sound level can be set higher when the driving mode is set on motor way driving compared to the sound level when the driving mode is city driving. Also the set of colors can be adapted, with a higher level of color alert depending on the speed. Also the text message can be adapted according to the speed of the vehicle.
The driver is informed that there is the potential risk which is identified and that he must change its position (or head position or look direction) in order to see the potential risk, by a text message (such as “change position to see danger in blind spot” or “blind spot danger”), the above mentioned color indication and sound indication or any combination thereof. If he does not change its position, the driver-assistance system will take over the control and will brake. Further, the driver-assistance may emit an external warning with the horn (or klaxon) of the vehicle 100 in order to alert the pedestrian 700.
In other words, the driver line of sight is masked by the front left pillar (blind spot area), thus the system may trigger a WARNING LEVEL 1. After a WARNING LEVEL 1, the system monitors driver attention to the risk. If the risk is in the blind spot line of sight of the driver, the specific warning (WARNING DANGER IN BLIND SPOT) is triggered to catch driver attention. If no action from the driver is performed to have the risk in his direct line of sight or even avoiding it, then a WARNING LEVEL 2 is triggered and the driver-assistance system feature can take control and brake to avoid collision with the pedestrian but at least the driver is aware of a danger in the blind spot and upcoming driver-assistance system action on vehicle motion.
It is of course understood that obvious improvements and/or modifications for one skilled in the art may be implemented, still being under the scope of the invention as it is defined by the appended claims.
In particular, reference is made to the use of another warning signal outside the vehicle 100.
| Number | Date | Country | Kind |
|---|---|---|---|
| 20210810 | Nov 2020 | EP | regional |
| Number | Name | Date | Kind |
|---|---|---|---|
| 8886023 | Satoh | Nov 2014 | B2 |
| 10604073 | Cho | Mar 2020 | B2 |
| 20120101701 | Moshchuk | Apr 2012 | A1 |
| 20160009175 | McNew | Jan 2016 | A1 |
| 20160332569 | Ishida | Nov 2016 | A1 |
| 20170247031 | Feit | Aug 2017 | A1 |
| Number | Date | Country |
|---|---|---|
| 108437898 | Apr 2019 | CN |
| 2168815 | Mar 2010 | EP |
| 3657467 | May 2020 | EP |
| 2009251761 | Oct 2009 | JP |
| 4946682 | Jun 2012 | JP |
| 20150044346 | Apr 2015 | KR |
| 101647730 | Aug 2016 | KR |
| 101992641 | Jun 2019 | KR |
| 2016209425 | Dec 2016 | WO |
| Entry |
|---|
| “Extended European Search Report”, EP Application No. 20210810.6, Jun. 7, 2021, 10 pages. |
| Lerner, et al., “Human Factors for Connected Vehicles: Effective Warning Interface Research Findings”, (Report No. DOT HS 812 068). Washington, DC: National Highway Traffic Safety Administration, Sep. 2014, 134 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20220169176 A1 | Jun 2022 | US |
