Patent Grant
12159615
This application claims priority from and the benefit of Korean Patent Application No. 10-2021-0166930, filed on Nov. 29, 2021, which is hereby incorporated by reference for all purposes as if set forth herein.
Exemplary embodiments of the present disclosure relate to an apparatus and method for controlling virtual engine sound, and more particularly, to an apparatus and method for controlling virtual engine sound, which use an annular array-shaped beam focusing directional speaker to change a sound focal point, thereby enabling the speaker to operate in a virtual engine sound system (VESS) mode or in a pedestrian communication mode in which a guidance sound is transmitted only to a specific position with pedestrians.
In general, an electric vehicle, such as an electric vehicle (EV), a hybrid electric vehicle (HEV), a plug-in hybrid electric vehicle (PHEV), or a fuel cell electric vehicle (FCEV), is an eco-friendly vehicle that may reduce exhaust gas and improve fuel consumption efficiency by combining an electric motor with an engine such as gasoline or diesel and allowing both power sources (electric motor and engine) to individually operate in an area where their respective characteristics may be exhibited depending on driving conditions
Such an electric vehicle generates no engine sound since it uses an electric motor as a power source while traveling at low speed, which causes the vehicle to generate very low noise. Hence, there is a risk that a safety accident may occur due to the inability of pedestrians (especially, who are visually impaired) to recognize approaching vehicles.
To improve this issue, a virtual engine sound system (VESS) has been developed that outputs a virtual engine sound, namely, a pedestrian protection sound, when a vehicle travels at low speed or reverses, thereby enabling pedestrians (blind, elderly, low-hearing people, etc.) to easily recognize the approach of vehicles. Moreover, many countries have made it mandatory for electric vehicles to be equipped with a virtual engine sound system (VESS) that artificially outputs engine noise.
Meanwhile, the performance of autonomous vehicles that have been recently released is improving.
If such autonomous vehicles have an advanced autonomous driving function intended to operate even in a city environment with intersections or the like, the interaction with pedestrians must also be taken into account. Accordingly, communication lighting has been introduced to mimic the process of non-verbal communication between pedestrians and drivers.
The communication lighting is a technology for lighting (for example, for lighting signs and images such as a crosswalk) on the road surface to indicate whether an autonomous vehicle recognizes a pedestrian and the intention of the autonomous vehicle to wait for a pedestrian to cross the road.
However, the communication lighting technology may have a lower communication effect to pedestrians depending on external lighting conditions since it is difficult to distinguish lighting areas on the road surface even when headlights are turned on during bright daytime. In addition, the communication lighting technology may cause an increase in cost to manufacture a vehicle because a new lighting device needs to be installed on the vehicle to irradiate signs or the like to be transmitted to pedestrians on the road surface. Moreover, the communication lighting technology may not cause pedestrians to understand the process of non-verbal communication in autonomous vehicles because vehicle manufacturers utilize different communication lighting methods.
Therefore, there is a need for a technology that enables interaction with pedestrians while performing its original function using a device or a system (e.g., VESS system) already installed in a vehicle without installing a separate device (e.g., lighting device for communication) in the vehicle.
The related art of the present disclosure is disclosed in Korean Patent No. 10-0638960 (published on Oct. 19, 2006, entitled “Method and Apparatus for Directing Sound).
Various embodiments are directed to an apparatus and method for controlling virtual engine sound, which use an annular array-shaped beam focusing directional speaker to change a sound focal point, thereby enabling the speaker to operate in a virtual engine sound system (VESS) mode or in a pedestrian communication mode in which a guidance sound is transmitted only to a specific position with pedestrians.
In an embodiment, there is provided an apparatus for controlling virtual engine sound, which includes a sensor unit configured to detect a position of a pedestrian in front of a vehicle, a speaker unit configured to output a virtual engine sound in all directions of the vehicle or output a sound guidance message only to the position of the pedestrian, and a control unit configured to cause the speaker unit to operate in either a virtual engine sound system (VESS) mode or a pedestrian communication mode in which the sound guidance message is transmitted only to the pedestrian's position, by changing a sound focal point output through the speaker unit.
The speaker unit may include an annular array-shaped beam focusing directional speaker.
The control unit may control an annular array-shaped beam focusing directional speaker to operate in the VESS mode to transmit the virtual engine sound in all directions, by forming the sound focal point inside the vehicle.
The control unit may adjust a delay time to transmit an output signal through each ultrasonic element of an annular array-shaped beam focusing directional speaker of the speaker unit in order to change the sound focal point.
In the VESS mode, the control unit may be configured to modulate the virtual engine sound with a specified sound modulation scheme to generate an output signal, to calculate a delay time to form the sound focal point inside the vehicle, and to apply the output signal to each ultrasound element of the speaker unit with a time difference corresponding to the delay time.
In the VESS mode, the control unit may adjust a level of the virtual engine sound depending on a vehicle speed.
In the pedestrian communication mode, the control unit may be configured to recognize the pedestrian's position, to modulate a specified guidance message with a specified sound modulation scheme to generate an output signal, to calculate a delay time to form the sound focal point in a pedestrian's ear, and to apply the output signal to each ultrasound element of the speaker unit with a time difference corresponding to the delay time.
The control unit may be configured to display the pedestrian's position as a bounding box by recognizing the pedestrian's position by a camera, to calculate a pedestrian's distance using the following Equation 1, and to calculate a top point of the bounding box that approximates a pedestrian's ear using the following Equation 2:
z=hcf/(0.5H−foot)p [Equation 1]
where hc is a height of the camera, f is a focal length, p is a pixel size, H is a height resolution of an image, and foot is a bottom height of the bounding box; and
y=hc+z(height−0.5H)p/f [Equation 1]
where height is a top height of the bounding box, hc is a height of the camera, f is a focal length, p is a pixel size, and H is a height resolution of an image.
In another embodiment, there is provided a method of controlling virtual engine sound, which includes detecting, by a control unit, a position of a pedestrian in front of a vehicle through a sensor unit, and causing, by the control unit, a speaker unit to operate in either a virtual engine sound method (VESS) mode or a pedestrian communication mode in which a sound guidance message is transmitted only to the pedestrian's position, by changing a sound focal point output through the speaker unit.
The control unit may control an annular array-shaped beam focusing directional speaker to operate in the VESS mode to transmit a virtual engine sound in all directions, by forming the sound focal point inside the vehicle.
The control unit may adjust a delay time to transmit an output signal through each ultrasonic element of an annular array-shaped beam focusing directional speaker of the speaker unit in order to change the sound focal point.
In the VESS mode, the control unit may be configured to modulate a virtual engine sound with a specified sound modulation scheme to generate an output signal, to calculate a delay time to form the sound focal point inside the vehicle, and to apply the output signal to each ultrasound element of the speaker unit with a time difference corresponding to the delay time.
In order for the speaker unit to operate in the VESS mode, the control unit may adjust a level of a virtual engine sound depending on a vehicle speed.
In order for the speaker unit to operate in the pedestrian communication mode, the control unit may be configured to recognize the pedestrian's position, to modulate a specified guidance message with a specified sound modulation scheme to generate an output signal, to calculate a delay time to form the sound focal point in a pedestrian's ear, and to apply the output signal to each ultrasound element of the speaker unit with a time difference corresponding to the delay time.
In order to recognize the pedestrian's position, the control unit may be configured to display the pedestrian's position as a bounding box by recognizing the pedestrian's position by a camera, to calculate a pedestrian's distance using the following Equation 1, and to calculate a top point of the bounding box that approximates a pedestrian's ear using the following Equation 2:
z−hcf/(0.5H−foot)p [Equation 1]
where hc is a height of the camera, f is a focal length, p is a pixel size, H is a height resolution of an image, and foot is a bottom height of the bounding box; and
y=hc+z(height−0.5H)p/f [Equation 2]
where height is a top height of the bounding box, hc is a height of the camera, f is a focal length, p is a pixel size, and H is a height resolution of an image.
As apparent from the above description, the apparatus and method for controlling virtual engine sound according to the present disclosure use the annular array-shaped beam focusing directional speaker to change the sound focal point, thereby enabling the speaker to operate in the virtual engine sound system (VESS) mode or in the pedestrian communication mode in which the guidance sound is transmitted only to the specific position with pedestrians.
Hereinafter, an apparatus and method for controlling virtual engine sound according to exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings.
It should be considered that the thickness of each line or the size of each component in the drawings may be exaggeratedly illustrated for clarity and convenience of description. In addition, the terms used herein are terms defined in consideration of functions of the present disclosure, and these terms may change depending on the intention or practice of a user or an operator. Therefore, these terms should be defined based on the entirety of the disclosure set forth herein.
As illustrated in
The sensor unit 110 includes at least one sensor (e.g., a camera sensor, a distance sensor, a LIDAR sensor, etc.) for detecting an obstacle in front of a vehicle.
The sensor unit 110 also detects a position of a pedestrian in front of the vehicle.
The speaker unit 130 includes an annular array-shaped beam focusing directional speaker.
For example, the speaker unit 130 is typically installed in a portion corresponding to an engine room of an internal combustion engine vehicle, but is not necessarily limited thereto.
The control unit 120 uses the annular array-shaped beam focusing directional speaker to change a sound focal point (i.e., note that it is not a directivity direction), thereby enabling the speaker to operate in a virtual engine sound system (VESS) mode or in a pedestrian communication mode in which a guidance sound is transmitted only to a specific position with pedestrians.
For example, the control unit 120 is able to control the annular array-shaped beam focusing directional speaker to operate in a VESS mode in which a warning sound (e.g., virtual engine sound) is transmitted in all directions, by forming a sound focal point inside the vehicle (for example, at the center of the speaker, inside a bumper, or the like). The control unit 120 is also able to control the speaker to operate in a pedestrian communication mode in which a guidance sound (i.e., sound guidance message) is transmitted only to a pedestrian's position (i.e., three-dimensional position) and a position of the pedestrian's ear (i.e., three-dimensional position), by detecting (calculating) those positions.
As such, the control unit 120 enables the speaker to operate in either the VESS mode or the pedestrian communication mode, thereby providing voice guidance only to a target pedestrian without an increase in city noise if necessary (e.g., by notifying a driver or an autonomous vehicle that a pedestrian is recognized). Accordingly, the pedestrian can clearly see whether the vehicle is stopped and can walk safely (for example, crossing a crosswalk, etc.).
In this case, since a delay time may be calculated by specification information or known technology of a speaker manufacturer, a detailed description of specific calculation thereof will be omitted in the present embodiment.
Hereinafter, the operation of the control unit 120 will be described with reference to
Referring to
In addition, the control unit 120 calculates a delay time (i.e., delay time to transmit an output signal through each ultrasonic element of an annular array-shaped beam focusing directional speaker) in order to form a beam focal point (i.e., sound focal point) inside the vehicle (e.g., at the center of the speaker, inside a bumper, or the like) (S103) (see
Furthermore, the control unit 120 uses the calculated delay time to apply the output signal to each ultrasound element of the annular array-shaped beam focusing directional speaker with a time difference corresponding to the delay time, thereby allowing the speaker to operate in the VESS mode in which a sound (i.e., virtual engine sound) is transmitted in all directions (S104).
As illustrated in
In this case, the control unit 120 may adjust the level of the warning sound (i.e., virtual engine sound) depending on the vehicle speed (S105).
On the other hand, if the vehicle is not traveling (No in S101), the control unit 120 recognizes a pedestrian's position (i.e., a three-dimensional position) (S106) (see
That is, the pedestrian's position (i.e., three-dimensional position) must be recognized because it is necessary to know the position of the pedestrian's ear in order to transmit a sound only to the pedestrian without causing noise in surrounding environments.
Referring to
z=hcf/(0.5H−foot)p [Equation 1]
where hc is the height of the camera, f is the focal length, p is the pixel size, H is the height resolution of the image, and foot is the bottom height of the bounding box.
Therefore, since the position of the pedestrian's ear may be calculated by the following Equation 2 since it approximates the top point of the bounding box:
y=hc+z(height−0.5H)p/f [Equation 2]
where height is the top height of the bounding box, hc is the height of the camera, f is the focal length, p is the pixel size, and H is the height resolution of the image.
Accordingly, if the lateral angle of view of the camera is given as design information, a lateral position may also be obtained from the ratio of the lateral resolution of the image to the lateral center point of the pedestrian's bounding box.
Therefore, by obtaining a distance from the bottom information of the pedestrian's bounding box, specifying the height of the ear from the top information, and obtaining the angle from the lateral center point, it is possible to obtain beam focal point (i.e., sound focal point) information for transmitting a sound only to the pedestrian who need guidance without causing noise to other pedestrians.
If the pedestrian is a pedestrian who needs guidance (for example, if the pedestrian is in a position to cross the road in front of the vehicle) (Yes in S107), the control unit 120 modulates a specified guidance message (e.g., “cross first”) with a specified sound modulation scheme (e.g., AM modulation scheme) to generate an output signal (S108).
For example, if the control unit 120 sets a beam focal point (i.e., sound focal point) to a pedestrian, the speaker is operable in a communication mode in which the intention of the autonomous vehicle is transmitted to the pedestrian. For example, assuming that sound is g(t), a signal p(t) output from each ultrasonic element is expressed by the following Equation 3:
p(t)=p0√1+mg(t)dt{tilde over ( )}sin(ω0t) [Equation 3]
where t is the time, w0 is the ultrasonic frequency, and P0 and m are constants.
The control unit 120 calculates a beam focal point (i.e., sound focal point) (e.g., top point of the bounding box that approximates the position of the pedestrian's ear) based on the pedestrian's bounding box information (S109).
In addition, the control unit 120 calculates a delay time (i.e., delay time to transmit an output signal through each ultrasonic element of an annular array-shaped beam focusing directional speaker) in order to form a beam focal point (i.e., sound focal point) in the pedestrian's ear (actually, at the top point of the bounding box that approximates the position of the pedestrian's ear) (S110) (see
Furthermore, the control unit 120 uses the calculated delay time to apply the output signal to each ultrasound element of the annular array-shaped beam focusing directional speaker with a time difference corresponding to the delay time, thereby allowing the speaker to operate in the pedestrian communication mode in which a sound (i.e., virtual engine sound) is transmitted to the pedestrian's ear (actually, at the top point of the bounding box that approximates the position of the pedestrian's ear) (S111).
As illustrated in
As described above, it is possible to use the annular array-shaped beam focusing directional speaker to change the sound focal point, thereby enabling the speaker to operate in the virtual engine sound system (VESS) mode or in the pedestrian communication mode in which the guidance sound is transmitted only to the specific position with pedestrians.
While the present disclosure has been described with respect to the embodiments illustrated in the drawings, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. It will be understood by those skilled in the art that various modifications and other equivalent embodiments may be made without departing from the spirit and scope of the disclosure as defined in the following claims. Therefore, the technical protection scope of the present disclosure should be defined by the following claims. The implementations described herein may be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed may also be implemented in other forms (e.g., an apparatus or program). The apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The method may be implemented in, for example, an apparatus such as a processor, which refers to a processing device in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes communication devices such as computers, cell phones, portable/personal digital assistants (“PDAs”), and other devices that facilitate communication of information between end-users.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2021-0166930 | Nov 2021 | KR | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 6229899 | Norris | May 2001 | B1 |
| 6694033 | Rimell | Feb 2004 | B1 |
| 10766479 | Galang | Sep 2020 | B2 |
| 11632644 | Bracht | Apr 2023 | B2 |
| 20080273722 | Aylward | Nov 2008 | A1 |
| 20080273723 | Hartung | Nov 2008 | A1 |
| 20090296954 | Hooley | Dec 2009 | A1 |
| 20120166042 | Kokido | Jun 2012 | A1 |
| 20130114827 | Lee | May 2013 | A1 |
| 20150127191 | Misra | May 2015 | A1 |
| 20150151680 | Park | Jun 2015 | A1 |
| 20160046234 | Kim | Feb 2016 | A1 |
| 20160052448 | Lee | Feb 2016 | A1 |
| 20160118037 | Yun | Apr 2016 | A1 |
| 20170129397 | Gee | May 2017 | A1 |
| 20170151907 | Sakamoto | Jun 2017 | A1 |
| 20190111839 | Lee | Apr 2019 | A1 |
| 20190385584 | Han | Dec 2019 | A1 |
| 20190392811 | Lee | Dec 2019 | A1 |
| 20210138959 | Soni | May 2021 | A1 |
| Number | Date | Country |
|---|---|---|
| 2010208439 | Sep 2010 | JP |
| 10-0638960 | Oct 2006 | KR |
| 20120077630 | Jul 2012 | KR |
| 20140080317 | Jun 2014 | KR |
| 20170006179 | Jan 2017 | KR |
| Entry |
|---|
| English Language Abstract of 10-0638960 published Oct. 19, 2006. |
| Number | Date | Country | |
|---|---|---|---|
| 20230169952 A1 | Jun 2023 | US |
