Patent Grant
11966052
The present application claims priority under 35 U.S.C. § 119 to Japanese Patent Application No. 2021-159822 filed on Sep. 29, 2021. The content of the application is incorporated herein by reference in its entirety.
The present invention relates to an alert system and a recording medium.
Conventionally, there has been proposed an image display system for a vehicle with which other vehicles traveling in the surroundings of the vehicle are detected by a camera or a radar, and marking images of other vehicles are displayed on a front window by a HUD (Head Up Display) to be superimposed on the other vehicles viewed by the driver (for example, see Japanese Patent Laid-Open No. 2017-021546).
As described above, when a target such as another vehicle existing in the surroundings of a mobile body such as the vehicle is detected, it is desired that the attention of the operator is promptly directed toward the target.
The present invention is designed in view of the aforementioned circumstances, and it is an object thereof to provide an alert system that supports the operator to promptly pay attention to the target and to provide a recording medium.
As a first aspect for achieving the foregoing object, there is an alert system including: a display device configured to output, to a front window of a mobile body, a visual display viewable by an operator of the mobile body;
In the alert system, the display control unit may: when the target is not recognized by the target recognition unit, set the display range of the surrounding icon as the whole circular area; and when a state where the target is not recognized by the target recognition unit shifts to a state where the target is recognized by the target recognition unit, gradually reduce the display range of the surrounding icon from the whole circular area to a part of the circular area corresponding to a direction from the mobile body toward the target.
In the alert system, when a state where the target of less than the upper limit value is recognized by the target recognition unit and a display mode of the surrounding icon is set to a first display mode that has a part of the circular area corresponding to the direction of the target with respect to the mobile body as the display range shifts to a state where the target is not recognized by the target recognition unit, the display control unit may be configured to switch the display mode of the surrounding icon to a second display mode that has the whole circular area as the display range.
In the alert system, the display control unit may perform processing for expanding a space between the mobile body icon and the surrounding icon once, while switching the display mode of the surrounding icon from the first display mode to the second display mode.
In the alert system, the display control unit may: when the target is not recognized by the target recognition unit, set the display mode of the surrounding icon to a third display mode that has the whole circular area as the display range; and when the state where the target is not recognized by the target recognition unit continues for a prescribed time or more, set the display mode of the surrounding icon to a fourth display mode that is different from the third display mode, with the whole circular area being the display range.
The alert system may include an influence degree recognition unit configured to recognize an influence degree of the target recognized by the target recognition unit for the mobile body, in which the display control unit may change a display mode of the surrounding icon according to the influence degree of the target.
The alert system may include a sound image localization control unit configured to change a sound image position of a prescribed alert sound perceived by the operator by controlling an output mode of a plurality of speakers provided in the mobile body, in which the sound image localization control unit may: when the target is not recognized by the target recognition unit, set the sound image position of the alert sound to a prescribed range of the surroundings of the operator; and when a state where the target is not recognized by the target recognition unit shifts to a state where the target is recognized by the target recognition unit, gradually reduce the range of the sound image position of the alert sound from the prescribed range toward a direction from the operator to the target.
As a second aspect for achieving the foregoing object, there is a non-transitory recording medium storing a control program for causing a computer to execute processing of: acquiring a recognition result of a target recognition unit that recognizes a target existing in surroundings of a mobile body; causing a controller that controls an operation of a display device to cause the display device to display a prescribed alert display according to the recognition result on a front window of the mobile body as a visual display, the display device outputting the visual display viewable by an operator of the mobile body on the front window; causing the display device to display, on the front window, the alert display including a mobile body icon representing the mobile body and a surrounding icon a display range of which is changed in a circumferential direction of a circular area surrounding the mobile body icon; when the target is recognized by the target recognition unit, causing the display device to display the surrounding icon on the front window by having a range including a part corresponding to a direction of the target with respect to the mobile body as the display range; when the number of targets recognized by the target recognition unit is a prescribed upper limit number or more, setting the display range of the surrounding icon to a whole circular area; and in a state where the number of the targets recognized by the target recognition unit is equal to or more than the upper limit number and the display range of the surrounding icon is set as the whole circular area, when the number of the targets recognized by the target recognition unit increases, reducing a size of the surrounding icon.
According to the alert system, it is possible to support the operator to promptly pay attention to the target.
By referring to
The controller 2 is a control unit that includes a processor 10, a memory 20, and an interface circuit or the like, not illustrated. The HUD 30 outputs, to a front window 40 of the vehicle 100, an alert display 50 that is a visual display viewable by a driver (operator) P illustrated in
As illustrated in
In addition to the alert system 1, the vehicle 100 also includes a plurality of cameras 31a to 31d, a radar 32, a plurality of sonars 33a to 33h, and a plurality of speakers 34a to 34d. As illustrated in
As illustrated in
The processor 10 functions as a target recognition unit 11, a display control unit 12, a sound image localization control unit 13, and an influence degree recognition unit 14 by reading and executing a control program 21 of the alert system 1 saved in the memory 20. The control program 21 may be read out from a non-transitory recording medium (a magnetic disk, an optical disk, a flash memory, or the like) and saved in the memory 20, or may be downloaded from an external server or the like and saved in the memory 20.
The target recognition unit 11 recognizes a target (pedestrian, another vehicle, or the like) existing in front of the vehicle 100 based on captured images acquired by the cameras 31a to 31d, an object position detection signal from the radar 32, and object position detection signals from the sonars 33a to 33h. While the embodiment describes a configuration that recognizes the target existing in the surroundings of the vehicle 100 by using the cameras 31a to 31d, the radar 32, and the sonars 33a to 33h, it is also possible to employ a configuration that recognizes the target not by using all of these but by using one of these or by a combination of two or more of these. Alternatively, it is also possible to employ a configuration that recognizes the target by other kinds of detection devices.
The display control unit 12 controls the operation of HUD 30 to control the output of the alert display 50 and to change the display mode of the surrounding icon 52. The sound image localization control unit 13 controls the output mode (output level, output timing, and the like) of the speakers 34a to 34d to change the sound image position of a prescribed alert sound perceived by the driver P. Note that there may simply need to be a plurality of speakers, and the number thereof may be three or less or five or more.
The influence degree recognition unit 14 recognizes the influence degree of the target for the vehicle 100 in four stages from level 0 to level 3 according to the recognized state of the target by the target recognition unit 11. Note that level 0 is a state where the target is not recognized by the target recognition unit 11. Levels 1 to 3 correspond to the state where the target is recognized by the target recognition unit 11, and the influence degree recognition unit 14 recognizes the influence degree by switching it from level 1 to level 2 and then to level 3, as the distance between the target and the vehicle 100 becomes shorter.
Note that the influence degree recognition unit 14 may recognize the influence degree by also considering the speed at which the target is approaching the vehicle 100. In this case, the influence degree recognition unit 14 recognizes the influence degree by switching it from level 1 to level 2 and then to level 3, as the speed at which the target is approaching the vehicle 100 becomes faster.
The display control unit 12 acquires the number of targets (target recognition result) recognized by the target recognition unit 11, and sets the alert level presented by the surrounding icon 52 and the alert sound in four stages from level 0 to level 3 based on the number of targets and the influence degree recognized by the influence degree recognition unit 14. When there is no target recognized by the target recognition unit 11, the display control unit 12 sets the alert level to level 0.
Furthermore, when the number of targets recognized by the target recognition unit 11 is one (a single target), the display control unit 12 sets the level of the influence degree of the target recognized by the influence degree recognition unit 14 as the alert level. For example, when the influence degree is level 2, the display control unit 12 sets the alert level to level 2.
Furthermore, when there is a plurality of targets recognized by the target recognition unit 11, the display control unit 12 increases the alert level as the number of targets increases. In the embodiment, the display control unit 12 sets the alert level to level 1 when the number of targets is two, sets the alert level to level 2 when the number of targets is three, and sets the alert level to level 3 when the number of targets is four or more. In this case, the number two corresponds to the upper limit number (threshold number) of the present disclosure.
When there are a plurality of targets recognized by the target recognition unit 11 and the influence degree of any one of the targets increases, the alert level may be increased. For example, in a state where there are two targets and the alert level is set to level 1, the alert level may be increased to level 2 from level 1 when the influence degree of one of the targets is increased to level 2.
The display control unit 12 sets the display mode (display range, diameter, color) of the surrounding icon 52 depending on the alert level according to the setting table illustrated in
By referring to the explanatory diagrams illustrated in
In Step S1 of
In a following Step S2, the display control unit 12 displays the surrounding icon 52 by the HUD 30 by having the display range thereof as the whole circular area. As illustrated in A11 of
In a next Step S3, the target recognition unit 11 searches a target existing in the surroundings of the vehicle 100. When the target existing in the surroundings of the vehicle 100 is recognized in a following Step S4, the target recognition unit 11 proceeds the processing to Step S20 of
In Step S5, the display control unit 12 starts a timer for checking continuation of the state where no target existing in the surroundings of the vehicle 100 is recognized. The set time of the timer is set to 3 seconds, for example. By looping the processing of subsequent Steps S6 to S8, the target recognition unit 11 searches the target existing in the surroundings of the vehicle 100 in Step S6 until the time in the timer is up in Step S8. Then, in Step S7, the target recognition unit 11 proceeds the processing to Step S20 of
In Step S9, the display control unit 12 switches the surrounding icon 52 to blinking display. The lighting mode of the surrounding icon 52 in this case corresponds to a fourth display mode of the present disclosure. The display control unit 12 blinks the surrounding icon 52 once in every several seconds, for example. This makes it possible to encourage the driver P to expand the caution range by not considering it as safe even when no target is recognized by the target recognition unit 11. In a next Step S10, the target recognition unit 11 searches the target existing in the surrounding of the vehicle 100. In a following Step S11, the target recognition unit 11 proceeds the processing to Step S20 of
The target recognition unit 11 proceeds the processing to Step S40 of
In a following Step S41, the display control unit 12 sets the display range of the surrounding icon 52 as the whole circular area 52r (see
The sound image localization control unit 13 executes processing of Step S50 in parallel to processing of Step S41 executed by the display control unit 12. In Step S50, the sound image localization control unit 13 outputs the alert sound according to the alert level with the sound image position being the whole surroundings of the driver P. In this case, the sound image localization control unit 13 performs processing such as increasing the volume of the alert sound or changing the tone of the alert sound such that the effect of the alert for the driver P becomes higher as the alert level is higher. Furthermore, the alert sound may be changed by superimposing a sound effect that helps imaging the attribute of the target (four-wheeled vehicle, two-wheeled vehicle, pedestrian, or the like) on the alert sound output in Step S1 of
In the meantime, in a case where there is a single recognized target, the target recognition unit 11 proceeds the processing to Step S21 from Step S20. In Step S21, the influence degree recognition unit 14 recognizes the influence degree of the target. In a following Step S22, the display control unit 12 determines the alert level according to the influence degree of the target, as described above.
In a next Step S23, the display control unit 12 sets the color and diameter of the surrounding icon 52 according to the alert level according to the setting table of
The sound image localization control unit 13 executes the processing of Step S30 and Step S31 in parallel to the processing of Steps S22 and S24. In Step S30, the sound image localization control unit 13 outputs an alert sound according to the alert level. In this case, the sound image localization control unit 13 outputs the alert sound with a still higher notification effect as the level of the influence degree of the target becomes higher. For example, the sound image localization control unit 13 may output an alert sound by superimposing a sound effect (onomatopoeia of engine sound or the like) that helps imaging the attribute of the target (four-wheeled vehicle, two-wheeled vehicle, pedestrian, or the like) on the alert sound that is started to be output in Step S1 of
In a next Step S31, the sound image localization control unit 13 gradually reduces the range of the sound image position 60 of the alert sound from the whole surroundings of the driver P illustrated in B1 toward a direction G that is from the driver P toward a target T as illustrated in B2 and B3 of
In Step S25, the target recognition unit 11 searches the target existing in the surroundings of the vehicle 100. In a following Step S26, the target recognition unit 11 proceeds the processing to Step S26 when the target is recognized, and proceeds the processing to step S1 of
In Step S27, the influence degree recognition unit 14 recognizes the influence degree of the target. In a following Step S28, the display control unit 12 determines the alert level according to the number of targets recognized by the target recognition unit 11 and the influence degree of the target recognized by the influence degree recognition unit 14. Then, the display control unit 12 changes the display mode of the surrounding icon 52 according to the change in the alert level. Furthermore, the sound image localization control unit 13 changes the output mode of the alert sound according to the alert level. Specifically, the display control unit 12 changes the display mode of the surrounding icon 52 as follows.
(1) When there is Change from Single Target to Plurality of Targets
As illustrated in
Then, the display control unit 12 expands the parts 52a and 52b corresponding to the directions of the two targets in the circumferential direction as illustrated in A23 so as to make the display range of the surrounding icon 52 to be the whole range of the circular area as illustrated in A24. As described, when the number of targets recognized by the target recognition unit 11 changes from a single target to a plurality of targets, it is possible to encourage the driver P to pay attention to a plurality of targets by expanding the display range of the surrounding icon 52 from the part corresponding to the direction of the target to the whole range of the circular area.
(2) When the Number of Targets Increases
When the number of targets recognized by the target recognition unit 11 increases, the display control unit 12 changes the display mode of the surrounding icon 52 according to the setting table illustrated in
As illustrated in
When the number of targets decreases, the display control unit 12 also changes the display mode of the surrounding icon 52 according to the setting table illustrated in
(3) When State where Target is Recognized Shifts to State where No Target is Recognized
When a state where a target is recognized by the target recognition unit 11 shifts to a state where no target is recognized by the target recognition unit 11, the display control unit 12 performs processing for moving the surrounding icon 52 away from the mobile body icon 51 once, as illustrated in
A41 in
Subsequently, the display control unit 12 makes the surrounding icon 52 in the size of “large diameter” that corresponds to level 0 of the alert level, and moves the surrounding icon 52 closer to the mobile body icon 51, as illustrated in A43. Then, the display control unit 12 expands the display range of the surrounding icon 52 toward the ambient direction of the surrounding icon 52 and sets the display mode to “large diameter, white” that corresponds to level 0, as illustrated in A44. The display mode of the surrounding icon 52 illustrated in A44 corresponds to a second display mode of the present disclosure. As described, by performing display by moving the surrounding icon 52 away from the mobile body icon once and then setting it to the display mode corresponding to level 0, it is possible to help the driver P intuitively perceive that the necessity of paying attention to the target is lowered.
In the embodiment above, as the alert processing for the driver P, executed are the processing of changing the display mode of the alert display 50 by the display control unit 12 and the processing of changing the sound image position of the alert sound by the sound image localization control unit 13. As another embodiment, the sound image localization control unit 13 may be omitted, and only the processing of changing the display mode of the alert display 50 by the display control unit 12 may be performed.
While the four-wheeled vehicle 100 is described in the embodiment above as a mobile body of the present disclosure, the mobile body of the present disclosure may be any mobile body operated by an operator, such as a two-wheeled vehicle, a flying object, a boat, or the like, for example.
In the embodiment above, when there is a single target recognized by the target recognition unit 11, the display control unit 12 sets the alert level according to the influence degree of the target recognized by the influence degree recognition unit 14 and changes the display mode of the alert display according to the setting table of the alert display illustrated in
In the embodiment above, when a state where no target is recognized by the target recognition unit 11 continues for a prescribed time or more, the display control unit 12 switches the alert display 50 from a lighting display (third display mode) to blinking display (fourth display mode) by the processing of Steps S5 to S9 illustrated in
While the HUD 30 is described in the embodiment above as the display device of the present disclosure, the display device of the present disclosure may have any configuration capable of outputting a visual display that can be viewed by the operator on the front window of the mobile body, and it may also be a display device other than a HUD.
While the size of the surrounding icon 52 is defined by the diameter of the surrounding icon 52 in the embodiment above, the size of the surrounding icon 52 may also be defined by the perimeter or the like of the surrounding icon 52. Furthermore, while the surrounding icon 52 is formed in a circular shape in the embodiment above, the surrounding icon 52 may be in any shape that surrounds the mobile body icon 51, such as an elliptical shape, a rectangular shape, or the like.
Note that
The above-described embodiments are specific examples of the following configurations.
(Configuration 1) An alert system including: a display device configured to output, to a front window of a mobile body, a visual display viewable by an operator of the mobile body; a display control unit configured to cause the display device to output a prescribed alert display to the front window as the visual display; and a target recognition unit configured to recognize a target existing in surroundings of the mobile body, in which the display control unit: causes the display device to display, on the front window, the alert display including a mobile body icon representing the mobile body and a surrounding icon a display range of which is changed in a circumferential direction of a circular area that surrounds the mobile body icon; when the target is recognized by the target recognition unit, causes the display device to display the surrounding icon on the front window by having a range including a part corresponding to a direction of the target with respect to the mobile body as the display range; when the number of the targets recognized by the target recognition unit is equal to or more than a prescribed upper limit number, sets the display range of the surrounding icon as the whole circular area; and in a state where the number of the targets recognized by the target recognition unit is equal to or more than the upper limit number and the display range of the surrounding icon is set as the whole circular area, when the number of the targets recognized by the target recognition unit increases, reduces a size of the surrounding icon.
According to the alert system of configuration 1, it is possible to support the operator to promptly pay attention to the target.
(Configuration 2) The alert system according to configuration 1, in which the display control unit: when the target is not recognized by the target recognition unit, sets the display range of the surrounding icon as the whole circular area; and when a state where the target is not recognized by the target recognition unit shifts to a state where the target is recognized by the target recognition unit, gradually reduces the display range of the surrounding icon from the whole circular area to a part of the circular area corresponding to a direction from the mobile body toward the target.
According to the alert system of configuration 2, it is possible to encourage the operator to pay attention to the direction where the target exists.
(Configuration 3) The alert system according to configuration 1 or 2, in which, when a state where the target of less than the upper limit value is recognized by the target recognition unit and a display mode of the surrounding icon is set to a first display mode that has a part of the circular area corresponding to the direction of the target with respect to the mobile body as the display range shifts to a state where the target is not recognized by the target recognition unit, the display control unit switches the display mode of the surrounding icon to a second display mode that has the whole circular area as the display range.
According to the alert system of configuration 3, it is possible to encourage the operator to pay attention in a wide range of the surroundings of the mobile body, when turned into a state where the target is not recognized.
(Configuration 4) The alert system according to configuration 3, in which the display control unit performs processing for expanding a space between the mobile body icon and the surrounding icon once, while switching the display mode of the surrounding icon from the first display mode to the second display mode.
According to the alert system of configuration 4, it is possible to help the operator intuitively perceive that the necessity of paying attention to the target is lowered.
(Configuration 5) The alert system according any one of configurations 1 to 4, in which the display control unit: when the target is not recognized by the target recognition unit, sets the display mode of the surrounding icon to a third display mode that has the whole circular area as the display range; and when the state where the target is not recognized by the target recognition unit continues for a prescribed time or more, sets the display mode of the surrounding icon to a fourth display mode that is different from the third display mode, with the whole circular area being the display range.
According to the alert system of configuration 5, when the state where the target is not recognized by the target recognition unit continues for the prescribed time or more, it is possible to encourage the operator to expand the caution range by not considering it as safe.
(Configuration 6) The alert system according to any one of configurations 1 to 5, including an influence degree recognition unit configured to recognize an influence degree of the target recognized by the target recognition unit for the mobile body, in which the display control unit changes a display mode of the surrounding icon according to the influence degree of the target.
According to the alert system of configuration 6, it is possible to notify the operator of the influence degree of the target for the mobile body by the difference in the display modes of the alert display.
(Configuration 7) The alert system according to any one of configurations 1 to 6, including a sound image localization control unit configured to change a sound image position of a prescribed alert sound perceived by the operator by controlling an output mode of a plurality of speakers provided in the mobile body, in which the sound image localization control unit: when the target is not recognized by the target recognition unit, sets the sound image position of the alert sound to a prescribed range of the surroundings of the operator; and when a state where the target is not recognized by the target recognition unit shifts to a state where the target is recognized by the target recognition unit, gradually reduces the range of the sound image position of the alert sound from the prescribed range toward a direction from the operator to the target.
According to the alert system of configuration 7, it is possible to increase the effect of directing the attention of the operator toward the target by gradually reducing the range of the sound image position of the alert sound toward the direction from the operator to the target.
(Configuration 8) A non-transitory recording medium storing a control program for causing a computer to execute processing of: acquiring a recognition result of a target recognition unit that recognizes a target existing in surroundings of a mobile body; causing a controller that controls an operation of a display device to control the display device to display a prescribed alert display according to the recognition result on a front window of the mobile body as a visual display, the display device outputting the visual display viewable by an operator of the mobile body on the front window; causing the display device to display, on the front window, the alert display including a mobile body icon representing the mobile body and a surrounding icon a display range of which is changed in a circumferential direction of a circular area surrounding the mobile body icon; when the target is recognized by the target recognition unit, causing the display device to display the surrounding icon on the front window by having a range including a part corresponding to a direction of the target with respect to the mobile body as the display range; when the number of targets recognized by the target recognition unit is a prescribed upper limit number or more, setting the display range of the surrounding icon to the whole circular area; and in a state where the number of the targets recognized by the target recognition unit is equal to or more than the upper limit number and the display range of the surrounding icon is set as the whole circular area, when the number of the targets recognized by the target recognition unit increases, reducing a size of the surrounding icon.
By causing the controller to execute the control program of configuration 8, it is possible to achieve the same operation effect as that of the alert system of configuration 1.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2021-159822 | Sep 2021 | JP | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 8676431 | Mariet | Mar 2014 | B1 |
| 10448180 | Helm | Oct 2019 | B1 |
| 20010020211 | Takayama | Sep 2001 | A1 |
| 20070273555 | Amano | Nov 2007 | A1 |
| 20120300072 | Kim | Nov 2012 | A1 |
| 20150002285 | Hatakeyama | Jan 2015 | A1 |
| 20160077688 | Shim | Mar 2016 | A1 |
| 20160264045 | Ng-Thow-Hing | Sep 2016 | A1 |
| 20170292848 | Nepomuceno | Oct 2017 | A1 |
| 20180001903 | Nagy | Jan 2018 | A1 |
| 20180105185 | Watanabe | Apr 2018 | A1 |
| 20180198955 | Watanabe | Jul 2018 | A1 |
| 20190204827 | Bhalla | Jul 2019 | A1 |
| 20200064983 | Higuchi | Feb 2020 | A1 |
| 20200101980 | Adams | Apr 2020 | A1 |
| 20210023994 | Watanabe | Jan 2021 | A1 |
| 20210239485 | Peng | Aug 2021 | A1 |
| 20210270620 | Nepomuceno | Sep 2021 | A1 |
| Number | Date | Country |
|---|---|---|
| 2017-021546 | Jan 2017 | JP |
| Number | Date | Country | |
|---|---|---|---|
| 20230113575 A1 | Apr 2023 | US |
