Patent Grant
11506906
The present application is based on PCT filing PCT/JP2019/036014, filed Sep. 13, 2019, which claims priority to JP 2018-199183, filed Oct. 23, 2018, the entire contents of each are incorporated herein by reference.
The present invention relates to a head-up display system that is mounted on a vehicle and suitably corrects a display position of an image.
Recently, an image display device for a vehicle (a head-up display (hereinafter, HUD)) that displays image information in front of the vehicle through a front window of the vehicle, as a virtual image, has been turned to practical use. At this time, information for a driver is provided as the image information to be displayed, and thus, a driving operation of the vehicle can be supported.
For example, in Patent Document 1, a configuration is proposed in which correction on a display position of a display image according to a vehicle vibration is described. Here, correction processing on a display position is executed for a weighted image (an image to be displayed to have a predetermined positional relationship with respect to a specific target in a real scene), and the correction processing is not executed for an unweighted image (an image to be displayed without having the predetermined positional relationship with respect to the target).
Patent Document 1: JP 2017-13590 A
In Patent Document 1 described above, “for example, includes a triaxial acceleration sensor or the like” is described as a vibration detection method, but how to perform the correction by using a sensor detection value is not described in detail. That is, according to the studies of the present inventors or the like, in a case where the correction for a pitching shake (a rotation shake) is performed by using a gyro sensor, a phenomenon has been found that the display image (the object) is moved in a direction having no relationship with the pitching shake in a curve travel. This seems strange to the driver, and thus, is required to be avoided. In addition, in a case where the correction processing is performed with a high accuracy, it is necessary to correct not only the pitching shake but also shake components (shift shakes) for an up-and-down direction. At this time, as a similar technology, for example, a camera stabilization method is known, but such a method is not capable of being directly applied, and it is necessary to perform the correction in consideration of a perspective position of the driver.
An object of the present invention is to provide a head-up display system that suitably corrects a display position of an image without seeming strange to a driver in relation to a vibration of a vehicle.
In a head-up display system of the present invention, a gyro sensor is installed in order to detect the vibration of a vehicle. An image data generation unit performs pitching correction on a display position of an object to be displayed, on the basis of angular velocity information for two axial directions acquired by the gyro sensor. In a case where the vehicle travels on a curve in an inclined state, the pitching correction is suppressed or stopped for a constantly displayed object to be displayed at a fixed position in an image display range. The brightness of the display of a real scene overlaid object to be displayed by being overlaid on a specific object detected by a forward sensing device is reduced or eliminated.
In addition, in a head-up display system of the present invention, a gyro sensor and an acceleration sensor are installed in order to detect a rotation shake and a shift shake as a vibration component of a vehicle. An image data generation unit performs correction on a display position of an object to be displayed, on the basis of angular velocity information acquired by the gyro sensor and acceleration information acquired by the acceleration sensor. A rotation radius in a case where it is assumed that a vertical displacement in the acceleration sensor occurs due to the rotation shake of the vehicle is obtained, the rotation shake and the shift shake of the vehicle at a position of a driver are calculated from information of a perspective position of the driver detected by the perspective detection device, and the correction on the display position of the object is performed.
According to the present invention, it is possible to provide a head-up display system that suitably corrects a display position of an image without seeming strange to a driver in relation to a vibration of a vehicle.
An embodiment of a head-up display system according to the present invention (hereinafter, an HUD system) will be described with reference to the drawings.
The HUD device 10 is mounted on a lower portion of a dashboard of the vehicle 2, and projects an image generated by an image display device onto a front window 3 (also referred to as a windshield) of the vehicle 2 through a mirror. The image reflected on the windshield 3 is incident on eyes 5′ of the driver, and thus, the driver is capable of visually recognizing the image. At this time, the driver is viewing a virtual image 8 in front of the windshield 3. A mirror driving unit 53 in the HUD device 10 axially rotates a mirror 52, in accordance with the height (A, B, and C) of the eyes 5′ of the driver, and adjusts a display position (a height direction) of the virtual image 8. According to such adjustment, the driver is capable of visually recognizing the virtual image 8 at an easily viewable position.
The perspective detection device 20, for example, is installed on the dashboard, and measures the position (the distance and the height) of the eyes 5′ of the driver. The perspective detection device 20 is used in a driver monitoring system (DMS). The forward sensing device 30, for example, is installed in an upper portion of the windshield 3, detects the forward object (a specific target) of the vehicle 2, and measures a distance to an object. The HUD device 10 determines an image to be displayed, on the basis of detection information of the perspective detection device 20 and the forward sensing device 30, and displays the image at an easily viewable position of the driver. Note that, the virtual image that is visually recognized by the driver will be also simply referred to as an “image 8”.
A CPU 13 of the HUD device 10 includes an image data generation unit 132 or a voice data generation unit 131, and generates image data or voice data to be provided to the driver, on the basis of the input vehicle information. The memory 14 stores a program executed by the CPU 13 or various control data items, and stores the image data to be displayed. The voice data generated by the voice data generation unit 131 is output to a speaker 54 through a voice interface 15. The image data generated by the image data generation unit 132 is displayed by an image display unit 50 through a display interface 16. The image display unit 50 includes a light source such as an LED or a laser, an image display device 51 generating image light by an illumination optical system and a display element such as a liquid crystal element, and the mirror 52 allowing the generated image light to exit toward the windshield 3. A mirror adjustment unit 133 in the CPU 13 adjusts the rotation of the mirror 52 through the mirror driving unit 53. A communication unit 134 is connected to the communication bus 61 through a communication interface 17, and transmits and receives detection data or control data with respect to the perspective detection device 20 or the forward sensing device 30.
A real scene overlaid object 81 is an object to be displayed by being overlaid specific object in a real scene (a vehicle, a person, a sign, and the like). Here, an alert (a ring) for attracting attention to a forward vehicle is displayed. In addition, as the real scene overlaid object 81, route information, a white line/breakdown lane, shop information (pop-up), and the like are displayed by being overlaid on a traveling path, a building, or the like.
A constantly displayed object 82 is an object to be displayed at a fixed position in an image display range 80 without being overlaid on the real scene. Here, velocity information is displayed at a bottom-right position of the image display range 80. In addition, as the constantly displayed object 82, residual fuel, an ambient temperature, destination information, and the like are displayed at each fixed position.
Note that, image data of the object to be displayed is generated by the image data generation unit 132 of the HUD device 10 with a 3D drawing library, and the display position thereof is set. In the case of displaying the real scene overlaid object 81, an object in the real scene 70 (a vehicle, a person, a sign, and the like) is sensed by the forward sensing device 30, and the position thereof is calculated. Then, the image data generation unit 132 draws an object to be overlaid on the sensed object.
Hereinafter, a correction method for the display image with respect to the vibration (the shake) of the vehicle will be described in accordance with cases. In Example 1, a pitching shake (a rotation component) is set to a target, and in Example 2, a pitching shake and a shift shake (a parallel movement component) are set to a target.
In Example 1, a case will be described in which a pitching shake (a rotation component) is set to a correction target, and correction processing is performed by using a gyro sensor.
Here, the present inventors or the like have found a phenomenon in which the image display range is moved in a direction having no relationship with the pitching shake in a curve travel.
Such a phenomenon will be considered. In a case where the vehicle body goes around an intersection or a curve in a state of being inclined in a right-and-left direction, the gyro sensor 43 detects an apparent pitch component by the following reasons. It is considered that this is because the object to be displayed is shifted in the up-and-down direction when the control unit 11 of the HUD device corrects the pitch component.
First, as illustrated in
The angular velocity ωpitch for the pitch direction becomes:
ωpitch=ω·sin θroll (1-1),
and
the component of ωpitch appears. In addition, the angular velocity ωyaw for the yaw direction becomes:
ωyaw=ω·cos θroll (1-2).
From Expressions (1-1) and (1-2),
ωpitch=ωyaw·tan θroll (1-3)
is obtained.
As described above, not only the angular velocity ωyaw for the yaw direction and the angular velocity ωpitch for the pitch direction set by the inclination θroll for the roll direction is detected by the gyro sensor. As a result thereof, the object to be displayed is shifted in the up-and-down direction when the control unit 11 of the HUD device corrects a pitch component ωpitch.
In order to avoid unnecessary display correction in the curve travel described above, [Coping Method 1] or [Coping Method 2] described below is implemented.
[Coping Method 1]
In Coping Method 1, a correction target is divided into a constantly displayed object and a real scene overlaid object, and the revision of a correction amount or the change of the brightness of the display is performed.
When the pitch angle detected by the gyro sensor 43 is θpitch, a pitch angle used in the correction is revised to θc by using Expression (2) for the constantly displayed object.
θc=θpitch·(1+exp(a(ωyaw+ω0)))−1·(1+exp(−−(ωyaw−ω0)))−1=θpitch·G (2)
Here, a damping term G in Expression (2) is a correction function shown in
In
Next, the correction for the real scene overlaid object 81 in
When the brightness of the object before the change is set to P, a brightness Pc after the change is revised by Expression (3).
Pc=P·(1+exp(a(ωyaw+ω0)))−1·(1+exp(−a(ωyaw−ω0)))−1=P·G(3)
Here, a damping term G in Expression (3) is the same function as the damping term G in Expression (2) described above, is asymptotic to G=1 in a case where the angular velocity |ωyaw| for the yaw direction is less than the threshold value ω0, and is asymptotic to G=0 in a case where |ωyaw| is greater than the threshold value ω0.
In
S101: The angular velocities (ωyaw and ωpitch) for the yaw direction and the pitch direction are acquired from the gyro sensor 43.
S102: Filter processing is implemented on the angular velocity ωpitch for the pitch direction. Specifically, offset is eliminated by HPF, and high-frequency noise is eliminated by LPF.
S103: ωpitch subjected to the filter processing is integrated, and thus, the pitch angle θpitch is calculated. Further, diffusion is prevented by multiplying a constant damping coefficient.
S104: The pitch angle θpitch is revised by Expression (2) described above, and θc is calculated. That is, in a case where the angular velocity |ωyaw| for the yaw direction is greater than the threshold value ω0, the pitch angle θc is brought close to 0.
S105: The brightness P of the object is revised by Expression (3) described above, and Pc is calculated. That is, in a case where the angular velocity |ωyaw| for the yaw direction is greater than the threshold value ω0, the brightness Pc of the object is brought close to 0.
S106: The image data generation unit 132 moves the position of the constantly displayed object, in accordance with the pitch angle θc after the revision.
S107: The brightness of the real scene overlaid object is changed in accordance with the brightness Pc after the revision. The object subjected to such correction is displayed by the image display unit 50.
S108: It is determined whether or not the display of the image is ended, and in a case where the display is continuously performed, the processing returns to S101, and the processing described above is repeated.
[Coping Method 2]
Next, Coping Method 2 will be described as a modification example of Coping Method 1. In Coping Method 2, the angular velocities for three axial directions is measured by the gyro sensor 43, a substantial pitching angle θc′ is calculated in consideration of the roll component ωroll of the angular velocity. In a pitch component ωpitch, a yaw component ωyaw, and a roll component ωroll of the angular velocity measured by the gyro sensor 43, the roll component ωroll is subjected to temporal integration to be a roll angle θroll.
An angular velocity ωc′ used in the pitching correction is obtained by the following expression.
ωc′=ωpitch−ωyaw·tan θroll (4)
In the angular velocity ωc′ obtained by Expression (4), the influence of the yaw component ωyaw and the roll angle θroll is considered, and thus, the angular velocity ωc′ is a substantial pitch component.
ωc′ described above is subjected to temporal integration, and the substantial pitch angle θc′ is obtained. The correction is performed by changing the position of the object to be displayed, in accordance with θc′. In this case, the same processing as that of the constantly displayed object is performed with respect to the real scene overlaid object without changing (eliminating) the brightness of the real scene overlaid object.
S111: The angular velocities (ωroll, ωyaw, and ωpitch) for the roll direction, the yaw direction, and the pitch direction are acquired from the gyro sensor 43.
S112: Filter processing is implemented on the angular velocities (ωroll, ωyaw, and ωpitch) for each of the directions. Specifically, offset is eliminated by HPF, and high-frequency noise is eliminated by LPF.
S113: ωroll subjected to the filter processing is integrated, and thus, the roll angle θroll is calculated. Further, diffusion is prevented by multiplying a constant damping coefficient.
S114: The angular velocity ωc′ is calculated from Expression (4) described above. At this time, the phase of ωpitch, ωyaw, and θroll is adjusted.
S115: The angular velocity ωc′ is subjected to temporal integration, and the substantial pitch angle θc′ is calculated. Further, diffusion is prevented by multiplying a constant damping coefficient.
S116: The image data generation unit 132 moves the position of the constantly displayed object and the real scene overlaid object, in accordance with the pitch angle θc′.
S117: It is determined whether or not the display of the image is ended, and in a case where the display is continuously performed, the processing returns to S111, and the processing described above is repeated.
Coping Method 1 and Coping Method 2 are compared. In Coping Method 1, it is not necessary to consider the roll direction, and thus, Coping Method 1 can be implemented by a biaxial gyro sensor. In the case of executing Coping Method 1, a user is capable of freely changing the threshold value ω0 and the coefficient a, in accordance with the preference. In Coping Method 2, a triaxial gyro sensor is required, and the object is constantly displayed on the basis of a substantial pitching angle.
In a case where the HUD device 10 is provided with a triaxial gyro sensor, the user is capable of selecting Coping Method 1 and Coping Method 2. In addition, Coping Method 1 and Coping Method 2 may be assigned in accordance with the type of real scene overlaid object. For example, in the case of a pop-up object including a character such as shop information, the user may gaze at the objects, and thus, Coping Method 1 may be assigned, and in the case of an object not including a character, Coping Method 2 may be assigned.
According to Example 1, a phenomenon can be avoided in which the object to be displayed is moved in a direction not having a relationship with the pitching shake when the vehicle travels on a curve. Accordingly, there is an effect in which when the vehicle exponentially converts the direction, the driver is prevented from being in an inattentive state, which contributes to the maintenance of safe driving.
In Example 1, the pitching shake has been treated as a correction target of the vibration of the vehicle, and in the vibration of the vehicle, there are the rotation component and the parallel movement component. In Example 2, both of the components will be treated. For this reason, the gyro sensor 43 and the acceleration sensor 42 are used as a shake sensor of the vehicle.
Such components are set to a correction target, and thus, the gyro sensor 43 and the acceleration sensor 42 are used, and a camera stabilization technology is applied. Here, in camera stabilization, a shake on an imaging surface is obtained, but a positional relationship between the sensor and the imaging surface is fixed, and thus, it is not necessary to consider the position of the driver. In contrast, in Example 2, it is necessary to obtain a shake at the position (the perspective) of the driver. At this time, the position of the driver is different in accordance with the body height or the body posture, and thus, is measured by using the perspective detection device 20 in real time.
The pitch component ωpitch of the angular velocity obtained from the gyro sensor 43 is integrated, and thus, the pitch angle θpitch is calculated. A distance between the gyro-acceleration sensors 43 and 42 and the driver 5 is set to L. A vertical displacement at the position of the gyro⋅acceleration sensors 43 and 42 is set to hg. A vertical displacement at the position of the driver 5 is set to hd. A rotation radius in a case where it is assumed that hg occurs due to the rotation shake is set to R. In the correction calculation, the pitch angle θpitch corresponding to a rotation shake amount and the displacement hd at the position of the driver 5 corresponding to a shift shake amount are obtained.
The following relational expression is established in each of the parameters.
hg=R·θpitch (here,θpitch<<1) (5-1)
dhg/dt(=Vz)=R·(dθpitch/dt) (5-2)
d2hg/dt2(=αz)=R·(d2θpitch/dt2) (5-3)
The rotation radius R is obtained by Expression (5-2) or (5-3), and is substituted into Expression (5-1), and thus, hg is obtained.
In the case of using Expression (5-2), a velocity dhg/dt is obtained by eliminating the gravitational acceleration from a z component (αz) of the acceleration sensor 42 with filter processing (HPF or the like), and by being subjected to temporal integration (set to Vz). dθpitch/dt is the pitch component (ωpitch) of the gyro sensor 43.
In the case of using Expression (5-3), an acceleration d2hg/dt2 is obtained by eliminating the gravitational acceleration from the z component (αz) of the acceleration sensor 42 with the filter processing. d2θpitch/dt2 is obtained by performing temporal differentiation with respect to the pitch component of the gyro sensor 43 (dωpitch/dt).
As described above, in a case where the values of each of the parameters hg, R, and L are obtained, the vertical displacement hd at the position of the driver 5 is calculated by Expression (5-4).
hd=hg·(R−L)/R (5-4)
In the correction on the display position of the object, the object is displayed by shifting the object position, in accordance with θpitch that is a rotation shake amount and hd that is a shift shake amount.
S201: The angular velocities (ωroll, ωyaw, and ωpitch) for three axial directions are acquired from the gyro sensor 43.
S202: The pitch component ωpitch is subjected to filter processing (offset is eliminated by HPF, and high-frequency noise is eliminated by LPF), and ωpitch subjected to the filter processing is integrated, and thus, the pitch angle θpitch is calculated (diffusion is prevented by multiplying a damping coefficient).
S203: ωpitch subjected to the filter processing is differentiated, and dωpitch/dt is calculated.
S204: The accelerations (αx, αy, and αz) for three axial directions are acquired from the acceleration sensor 42.
S205: The acceleration αz of the vertical component is subjected to filter processing (the gravitational acceleration is eliminated by HPF, and high-frequency noise is eliminated by LPF), and is integrated, and thus, the velocity Vz for the vertical direction is calculated (diffusion is prevented by multiplying a damping coefficient).
S206: The rotation radius R and the vertical displacement hg of the sensor are calculated by using αz, Vz, ωpitch, and dωpitch/dt obtained as described above. At this time, R is calculated from Expression (5-2) or (5-3), and hg is calculated from Expression (5-1).
S207: The distance L between the gyro⋅acceleration sensors 43 and 42 and the driver 5 is measured by the perspective detection device 20.
S208: The vertical displacement hd of the driver 5 is calculated from Expression (5-4) by using R, hg, and L obtained as described above.
S209: The display position of the object is changed in accordance with the rotation shake amount θpitch and the shift shake amount hd.
S210: It is determined whether or not the display is ended, and in a case where the display is continuously performed, the processing returns to S201, and the processing described above is repeated.
Here, in Example 2, the following modification example can be included. In the modification example, the gyro sensor 43 and two acceleration sensors 42 are installed, and the rotation radius is obtained by another method.
The following relational expression is established in each of the parameters.
hga=Ra·θpitch (here,θpitch<<1) (6-1a)
hgb=Rb·θpitch (here,θpitch<<1) (6-1b)
dhga/dt=Ra·(dθpitch/dt) (6-2a)
dhgb/dt=Rb·(dθpitch/dt) (6-2b)
d2hga/dt2=Ra·(d2θpitch/dt2) (6-3a)
d2hgb/dt2=Rb·(d2θpitch/dt2) (6-3b)
Ra−Rb=D (6-4)
The rotation radii Ra and Rb are obtained from Expressions (6-2a) and (6-2) or Expressions (6-3a) and (6-3b).
First, in the case of using Expressions (6-2a) and (6-2b),
(dhga/dt)/Ra=(dhgb/dt)/Rb (6-5)
Here, in a case where a velocity component is set such that (dhga/dt)=Va and (dhgb/dt)=Vb, from Expressions (6-4) and (6-5),
Ra=D·Va/(Va−Vb) (6-6a)
Rb=D·Vb/(Va−Vb) (6-6b)
are obtained, and the rotation radii Ra and Rb are obtained.
On the other hand, in the case of using Expressions (6-3a) and (6-3b),
(d2hga/dt2)/Ra=(d2hgb/dt2)/Rb (6-7)
Here, in a case where an acceleration component is set such that (d2hga/dt2)=αa and (d2hgb/dt2)=αb, and from Expressions (6-4) and (6-7),
Ra=D·αa/(αa−αb) (6-8a)
Rb=D·αb/(αa−αb) (6-8b)
are obtained, and the rotation radii Ra and Rb are obtained.
Note that, the positive or negative sign of the rotation radii Ra and Rb are set in accordance with a positional relationship between the acceleration sensors 42a and 42b and the pivot point S. From Expressions (6-1a) and (6-1b), in a case where the polarities (the directions) of the θpitch and the displacement hgx (x: a and b) are the same, Rx (x: a and b) is positive, and in a case where the polarities are different, Rx is negative. In
As described above, in a case where the value of each of the parameters hga, Ra, and L is obtained, the vertical displacement hd at the position of the driver 5 is calculated from Expression (6-9).
hd=hga·(Ra−L)/Ra (6-9)
In the correction for the object, the object is displayed by shifting the object position, in accordance with the rotation shake amount θpitch and the shift shake amount hd. According to the modification example described above, it is not necessary to use the value of the gyro sensor 43 in order to obtain the rotation radius R.
According to Example 2, the display position of the object to be displayed can be corrected with a high accuracy by detecting both of the rotation shake and the shift shake, as the vibration of the vehicle, and by considering the perspective position of the driver.
| Number | Date | Country | Kind |
|---|---|---|---|
| JP2018-199183 | Oct 2018 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2019/036014 | 9/13/2019 | WO |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2020/084954 | 4/30/2020 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 20060041367 | Ono | Feb 2006 | A1 |
| 20170038595 | Kutomi | Feb 2017 | A1 |
| 20180024354 | Shibata | Jan 2018 | A1 |
| 20190139286 | Shimoda | May 2019 | A1 |
| 20190333481 | Hato et al. | Oct 2019 | A1 |
| 20190359228 | Banno et al. | Nov 2019 | A1 |
| Number | Date | Country |
|---|---|---|
| 2011-112591 | Jun 2011 | JP |
| 2013237320 | Nov 2013 | JP |
| 2015-63178 | Apr 2015 | JP |
| 2017-13590 | Jan 2017 | JP |
| 2018-77400 | May 2018 | JP |
| 2018127204 | Aug 2018 | JP |
| 2018-140714 | Sep 2018 | JP |
| WO-2018042898 | Mar 2018 | WO |
| Entry |
|---|
| International Search Report and Written Opinion dated Nov. 26, 2019, received for PCT Application No. PCT/JP2019/036014, Filed on Sep. 13, 2019, 8 pages including English Translation. |
| Notice of Reasons for Refusal has been issued for the corresponding Japanese patent application No. 2020-552964 dated Apr. 19, 2022. |
| Number | Date | Country | |
|---|---|---|---|
| 20210389596 A1 | Dec 2021 | US |
