Patent Application
20240121555
The present invention relates to an audio system of a motor vehicle.
A human driver of a motor vehicle may want to have a conversation with another passenger while he is driving. A problem is that the audio system of the vehicle may be playing, and its volume may be high enough to make it difficult for the driver and passenger to hear each other talk. In this event, the driver typically must manually turn down the volume of the audio system, which diverts his attention from the driving task. Further, when the conversation has ended, the driver must manually turn the audio volume back up.
The present invention may improve the user experience by automatically ducking (i.e., reducing) the volume of the audio system when conversation is detected between two seating zones. By using zoned audio, beamforming and speech detection, the present invention can reduce user engagement when driving by automatically ducking the audio volume levels.
A motor vehicle of the present invention may include an integrated microphone, may be capable of detecting multizone audio, and may be capable of performing beamforming. That is, the vehicle may have a respective microphone for each seat and/or for each seat zone. The microphones may have beamforming capability, enabling the in-vehicle infotainment (IVI) to detect from which seat the speech originated.
The invention comprises, in one form thereof, an audio system for a motor vehicle having a passenger compartment. The system includes a plurality of microphones disposed in the passenger compartment and each producing a respective microphone signal indicative of audible voices in the passenger compartment. A plurality of loudspeakers are disposed in the passenger compartment and emit sounds based on infotainment audio signals. An audio processor determines from the microphone signals respective locations of people in the passenger compartment who are participating in a voice conversation. The audio processor modifies the infotainment audio signals sent to individual ones of the loudspeakers based on the determined locations such that the sounds emitted by the loudspeakers are quieter as heard by the people participating in the voice conversation than as heard by at least one other person in the passenger compartment.
The invention comprises, in another form thereof, a method for controlling an audio system of a motor vehicle, including using a plurality of microphones disposed in the passenger compartment to each produce a respective microphone signal indicative of audible voices in the passenger compartment. A plurality of loudspeakers disposed in the passenger compartment are used to each emit sounds based on infotainment audio signals. Based upon the microphone signals, respective locations of people in the passenger compartment who are participating in a voice conversation are determined. The infotainment audio signals sent to individual ones of the loudspeakers are modified based on the determined locations such that the sounds emitted by the loudspeakers are quieter as heard by the people participating in the voice conversation than as heard by at least one other person in the passenger compartment.
The invention comprises, in yet another form thereof, an audio system for a motor vehicle having a passenger compartment. The system includes a plurality of microphones disposed in the passenger compartment. Each microphone is associated with a respective seat in the passenger compartment and produces a respective microphone signal indicative of audible voices in the passenger compartment. A plurality of loudspeakers are disposed in the passenger compartment. Each loudspeaker is associated with a respective seat in the passenger compartment and emits sounds based on infotainment audio signals. An audio processor determines from the microphone signals which seats are being sat in by people who are participating in a voice conversation. The audio processor modifies the infotainment audio signals sent to individual ones of the loudspeakers based on which of the seats are determined to be sat in by the people who are participating in the voice conversation. The infotainment audio signals are modified such that the sounds emitted by the loudspeakers associated with the seats determined to be sat in by the people who are participating in the voice conversation are of a lower volume than sounds emitted by ones of the loudspeakers associated with ones of the seats being sat in by people who are not participating in the voice conversation.
The above-mentioned and other features and objects of this invention, and the manner of attaining them, will become more apparent and the invention itself will be better understood by reference to the following description of embodiments of the invention taken in conjunction with the accompanying drawings, wherein:
The embodiments hereinafter disclosed are not intended to be exhaustive or limit the invention to the precise forms disclosed in the following description. Rather the embodiments are chosen and described so that others skilled in the art may utilize its teachings.
In a first step 102, speech in the vehicle is detected. Speech and silence detection technologies for detecting speech are well known to those of skill in the art. An audio processor may analyze microphone signals and determine that the signals include frequencies and volume consistent with human speech.
In a next step 104, it is determined whether the detected speech is directed towards another passenger. When speaking to another passenger, at least during the beginning of the conversation, a vehicle occupant typically orients their speech/action in such a way that the speech is directed toward the other occupant that they are talking to. A vehicle occupant (other than the driver) may either turn around or look sideways towards another passenger. A driver tends to see the passenger by looking at them either side ways or through the rearview mirror. The person they are looking at can be detected by using beam formers and/or an in-vehicle camera. Using both beam formers and an in-vehicle camera may result in best performance.
If there is no in-car camera, which makes facial detection impossible, then the occupant may be detected using beam formers only. Occupant detection using beam formers may rely on the audio power delivered to different zone microphones. When the driver speaks to the vehicle for wake up word (WuW) and similar speech recognition technology-related action, they tend to look at the road ahead between short peeks at the head unit's main screen to see what action has been taken. On the other hand, when talking to another passenger, the driver tends to either see a rear seat passenger in the mirror. Else, the driver may slightly tilt their head while talking to the passenger. Either way, this results in lower audio power being generated for the driver's microphone, but a slightly higher power being picked up by the front passenger microphone and/or rear microphones.
In the case of facial recognition and action detection, using available machine learning technologies, it can be detected whether the driver is speaking, looking sideways, responding to passengers' comments, etc.
Next, in step 106 the audio volume level is reduced (i.e., ducked). If speech is directed towards a particular passenger, then the background music volume levels are reduced for only the two passenger zones/seats of the two people involved in the conversation. Zoned audio may enable users to isolate audio to individual seats to the greatest extent. When audio volume is ducked for particular seats, users of the vehicle may be able to have seamless conversations without having to engage with in-vehicle infotainment systems or having to disrupt the music experiences of other passengers.
In step 108, it is detected that the user's conversation has ended. Again, speech and silence detection technologies for detecting speech and the lack of it are well known to those of skill in the art.
In a final step 110, at the end of the users' conversation, the volume is returned back to the normal level. Thus, the invention results in an improvement in the form of reduced user action since the user does not need to touch radio buttons to make volume adjustments, which makes driving safer. Another advantage is that user annoyance is reduced as natural human behavior is accommodated, and the user does not need to take action in order to talk to another passenger, thereby improving the user experience.
During use, microphones 1-n1 transmit signals 206 to audio processor 202 indicative of the sounds picked up by the microphones, as is conventionally known. Based on these signals, processor 202 determines which seats/zones associated with microphones 1-n1 are occupied by people participating in a conversation. For example, processor 202 may determine that the loudest voices are being detected by microphones 2 and 4 (and possibly that both microphones 2 and 4 are each detecting two different loud voices), and thus the occupants of seats/zones 2 and 4 are having a conversation. Additionally, video processor 214 may transmit signals 216 to audio processor 202 to indicate that images captured by video cameras 1-n3 indicate that one or more of the people involved in the conversation are looking at at least one other participant in the conversation, thereby indicating or confirming that a conversation between passengers is taking place.
So that the people having the conversation can hear each other better, processor 202 may transmit signals 212 to control amplifier 204 to transmit signals 208 to speakers 2 and 4 that cause the audio volume of the music or other content being played by speakers 2 and 4 to be reduced. Amplifier 204 may transmit reference signal 210 to audio processor 202 to inform processor 202 about the current state of signals 208 being sent to the speakers. Once processor 202 determines, based on the signals 206 from the microphones, that the conversation has ended, processor 202 may cause amplifier 204, via signals 212, to transmit signals 208 to speakers 2 and 4 that cause the audio volume of the music or other content being played by speakers 2 and 4 to be increased back to the previous level that was in effect before the conversation was detected.
Next, in step 304, a plurality of loudspeakers disposed in the passenger compartment are used to each emit sounds based on infotainment audio signals. For example, loudspeakers 1-n2 are disposed in the passenger compartment and may emit sounds based on infotainment audio signals 208.
In a next step 306, it is determined, based upon the microphone signals, respective locations of people in the passenger compartment who are participating in a voice conversation. For example, processor 202 may determine that the loudest voices are being detected by microphones 1 and 3 (and possibly that both microphones 1 and 3 are each detecting two different loud voices), and thus the occupants of seats/zones 1 and 3 are having a conversation.
In a final step 308, the infotainment audio signals sent to individual ones of the loudspeakers are modified based on the determined locations such that the sounds emitted by the loudspeakers are quieter as heard by the people participating in the voice conversation than as heard by at least one other person in the passenger compartment. For example, so that the people having the conversation can hear each other better, processor 202 may transmit signals 212 to control amplifier 204 to transmit signals 208 to speakers 1 and 3 that cause the audio volume of the music or other content being played by speakers 1 and 3 to be reduced.
The invention has been described as lowering the audio volume of all frequencies for people involved in a conversation. However, in another embodiment, only the loudspeaker frequencies matching the frequencies of human voices, or perhaps matching the frequencies of the voices of the particular people involved in the conversation, have their volumes lowered.
While this invention has been described as having an exemplary design, the present invention may be further modified within the spirit and scope of this disclosure. This application is therefore intended to cover any variations, uses, or adaptations of the invention using its general principles. Further, this application is intended to cover such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains.
This application claims benefit of U.S. Provisional Application No. 63/413,855, filed on Oct. 6, 2022, the disclosure of which is hereby incorporated by reference in its entirety for all purposes.
| Number | Date | Country | |
|---|---|---|---|
| 63413855 | Oct 2022 | US |
