Patent Application
20250218435
Exemplary embodiments of the invention relate to a method for processing speech inputs which are used to control vehicle functions in a vehicle, as well as to an operating device for controlling vehicle functions having at least one microphone and one screen.
An operating system for a vehicle is fundamentally known from DE 103 60 655 A1. The disclosed operating system comprises a display for displaying entries in a menu structure. The entries in the menu structure can be activated via speech control, wherein at least one entry from the menu structure simultaneously forms a keyword for the speech control.
Vehicle functions are increasingly able to be operated in a vehicle via speech control. In the case of the mentioned patent publication, the commands are formed by keywords within the menu, so that it is possible to recognize on the screen which command can be used to activate which vehicle function.
Furthermore, it is fundamentally known in the field of speech control how to control different vehicle functions via speech. Here, a distinction must essentially be made between two different types of command. One type is the command referred to as an unambiguous command in the scope of this disclosure, the structure of which command indicates which function should be controlled. For example, the command words “navigate to” unambiguously address a vehicle function in the area of the navigation system, such as inputting an address and then navigating to this address. In the case of an incoming telephone call, an unambiguous command would be “accept call” or “reject call”, for example. The command itself can therefore be unambiguously assigned to a specific vehicle function.
Furthermore, there is the command, referred to as a neutral short command in the scope of this disclosure, which is ambiguous. The neutral short command is frequently only formed by a word, e.g., in the German or also in the English language. Therefore, for example, the command “start” can mean both starting a navigation system, starting a certain playlist in the media system, the activation of seat heating, the activation of lighting, or optionally also accepting a phone call or similar. If different vehicle functions are available, it cannot therefore be defined exactly via such a neutral short command which vehicle function should be started via the short command because of its ambiguous nature. If different vehicle functions are available that can be activated via the neutral short command, these can be processed sequentially one after the other, for example. However, this is not particularly convenient for the user and, in case of doubt, can lead to the actually desired command being processed with a delay and vehicle functions possibly being started that the person using the vehicle did not actually intend to be started.
Exemplary embodiments of the present invention are therefore directed to an improved method enabling an efficient and targeted processing of speech commands for controlling vehicle functions even when neutral short commands are used, as well as to an operating device suitable for carrying out this method.
The method according to the invention for processing speech inputs, which are used to control vehicle functions in a vehicle, comprises speech recognition and graphics assigned to the vehicle functions, which are displayed on a screen of the vehicle. Different vehicle functions, which are available at the current time for pending activation by speech, can therefore be displayed simultaneously on one screen, for example in the style of boxes arranged next to each other or other types of entries. According to the invention, one of the graphics is now prioritized and distinctively marked on the screen. This displayed marking shows a user which function will react directly to an issued neutral short command, such as for example the word “start” or “launch”, preferably without mentioning a keyword, with the other vehicle functions symbolized by the other graphics displayed at the same time not reacting directly to such a neutral short command, which in principle would also be suitable for activating these vehicle functions. The neutral short command thus cannot be unambiguously related to a vehicle function or a category comprising a vehicle function, i.e., it cannot be determined using the short command which vehicle function this refers to. The short command, preferably consisting of a word, is to be understood to mean here that the command does not comprise information about a desired function, for example by a keyword describing the function or making a reference, but can be used universally for different applications. The distinctive marking of that displayed graphic out of the multiple simultaneously displayed graphics which is assigned to the vehicle function, which directly reacts to the neutral short command following the display of the graphics, means that a person using the vehicle can recognize with a quick look at the screen which of the vehicle functions currently can be controlled directly by the neutral short command. Preferably, the marking of each graphic assigned to the function reacting to a short command and shown on a display surface is always the same, consistent or at least equivalent.
In a very advantageous further development of the method according to the invention the vehicle function reacting to the neutral short command is determined from predefined functions suitable for the selection with a neutral command, wherein in each case one function of the predefined functions is dynamically prioritized with respect to a reaction to the neutral short command. Such predefined functions can be created, for example, using the empirical experience of the software developer for the vehicle, for example in the form of a list. The vehicle functions suitable with a short command are the applications that can be represented by the graphics, the function of which can be activated by a short command, i.e., preferably by an individual word. The prioritization of one of the predefined functions means that this function, i.e., currently only this function, can be activated with a neutral short command or reacts to the neutral short command. The dynamic prioritization makes it possible to determine the function reacting to the neutral short command depending on the situation, based on boundary conditions or sensor data read in from the vehicle. The prioritized function or the graphic assigned to the function is provided with a marking. The marking preferably looks identical or equivalent in terms of appearance for each of the prioritized functions, so that a user immediately recognizes the function that can be operated with a neutral speech command. The read-in sensor data comprises, for example, information about the driving status of the vehicle, about the driver and their status or about environmental conditions, which are either determined in the vehicle itself or are made available via a server via telematics. For example, a status change of speech-operated functions, such as an incoming telephone call, causes the priority to be dynamically shifted to the function whose status is changing. As further examples for status changes that can be reacted to with a speech input, mention could be made of appearing pop-up notifications of an information system, warning messages, traffic notifications or reaching a geographical position.
Additionally, or in particular alternatively, the prioritized vehicle function reacting to the neutral short command is determined by means of a model trained with historical data based on its relevance for a person using the vehicle. For example, a person who frequently uses the vehicle and who has repeatedly called up certain vehicle functions in certain situations can be taken into account accordingly, so that in a similar situation, the vehicle function that is called up particularly frequently in such a situation is given the highest priority and accordingly becomes the marked function that reacts directly to the neutral short command. Such models trained with historical data can be based on methods of adaptive learning or, in particular, the use of artificial intelligence or neural networks. The personalized prioritized functions are stored in a personalized manner.
As already mentioned, the vehicle function reacting to the neutral short command can also be determined, in particular, based on the vehicle context, for example the time of day, the time of year, the ambient temperature, the ambient brightness, the driving situation, a specific environmental situation detected by the vehicle's environment sensors, or a change in the status of a speech-operated function such as a telephone or an information system.
A person who sets up the vehicle determines and stores in a control unit which function is prioritized based on the vehicle context based on experience or expectation, for example activating the seat heating shortly after starting the vehicle in cold ambient temperatures.
A further additional or alternative way of determining the vehicle function reacting to the neutral short command, preferably prioritizing this as such from a selection of functions, is to determine it based on the person using the vehicle. For example, a profile can be called up for the corresponding person, who is recognized, for example, by an individually assigned vehicle key, a mobile end device assigned to them, such as a smartphone or similar. If this person is the person using the vehicle, in particular the person controlling the vehicle, the vehicle functions that can be called up directly via the neutral short command can, for example, be prioritized based on a priority stored in this person's profile or their individually set and saved preferences.
In addition, according to an extraordinarily favorable further development of the method according to the invention, in the event that by means of a speech input having an unambiguous command which thus comprises the direct or indirect naming of one of the vehicle functions which do not react to the neutral short command, in particular independently of the display on the screen. For example, if the vehicle function for starting the seat heating in the current situation is the one on the screen that would react to the neutral short command, then a different vehicle function can be activated at any time with an unambiguous command such as “navigate home”. In this case, navigation is started accordingly. The unambiguous command comprises a keyword that can be unambiguously assigned to a vehicle function. The naming of the unambiguous command preferably activates the function independently of the display on the screen, i.e., even if no graphic assigned to the navigation is displayed on the screen.
According to another very favorable additional embodiment, one of the graphics is distinctively marked by an operating action of a person using the vehicle, so that the vehicle function assigned to this graphic is then specified as the vehicle function reacting to the neutral short command. A person using the vehicle can therefore override the presetting of the vehicle function reacting to the short command, which has been carried out in accordance with the methods described above, for example by selecting this function via a look or pointing gesture to the graphic or to an element unambiguously assigned to the vehicle function or else by touching the corresponding graphic on a touch-sensitive screen in order to react to the neutral short command. This means that the suggestion made by the control devices of the vehicle can be overridden so that changes can be made easily and efficiently. Such changes can be used again in the embodiment variant described above, in which the historical experiences of the system are taken into account, in order to learn for similar situations that the vehicle function selected by the operator action is automatically specified in the future in comparable situations or is prioritized higher accordingly in a priority list.
A very advantageous embodiment provides that the graphic assigned to the vehicle function reacting to the neutral short command is displayed on the screen by means of a graphic highlighting and/or centering in a horizontal and/or vertical direction on the screen. The graphic is therefore emphasized accordingly. This can take place by a simple marking, such as for example a marking symbol in the form of a dot or circle, but it can also be implemented by various other graphic possibilities. For example, the highlighted graphic can be changed in terms of its color or size, it can be backlit accordingly, it can flash or move to make it clear that this graphic or the vehicle function assigned to this graphic is the one that can be selected directly using the neutral short command. As an alternative or, in particular in addition to such graphic highlighting, the highlighting can also be realized by centering, for example, in a horizontal and/or vertical direction on the screen. For example, the graphic is displayed as the central graphic on the screen, while the other graphics, which symbolize other vehicle functions, move to the right and left or top and bottom edges.
According to a very advantageous embodiment of the method according to the invention, it can further be provided that the distinctive marking changes from the one graphic to another graphic as soon as the vehicle function assigned to the other graphic becomes the one that can be addressed via the neutral short command. So, if there is a change in the assignment, the marking can change from the one graphic, preferably in an animated display, to the other graphic. The trigger for the change can be a change in prioritization due to a changed situation, for example. For example, various vehicle functions can be displayed statically. For example, starting the seat heating can be selected as the vehicle function that can be addressed via the neutral short command and the relevant graphic can be visually marked accordingly. If the situation changes, for example because a call comes in, this incoming call is typically given higher priority in the current situation, so that the graphic for the incoming call is highlighted accordingly so that the call can be answered easily and efficiently using the neutral short command, for example “start”. The same can apply to other situations, for example if a traffic jam has been recognized by the navigation device and a modified route suggestion has been created, which is offered to the person using the vehicle and in this situation becomes the vehicle function that should be able to be started easily and efficiently via the neutral short command. The animated representation comprises, for example, a continual gliding over of a marking of a graphic of one vehicle function onto a graphic of another vehicle function.
In an advantageous embodiment of the method, information about a speech input request from a user relating to a function displayed on a screen by means of the graphic is transmitted to a speech dialogue system together with the priority assigned to the vehicle function and, if the speech dialogue system confirms the priority, the graphic associated with the vehicle function is marked as reacting to the neutral short command. The request for a speech input to be made by a user is a graphic associated with the function on the display surface, which can be controlled by means of a speech input from a user and preferably asks for a speech input. Information about this preferably upcoming request is transmitted together with prioritization information to the speech dialogue system, which is thus activated for an evaluation of an upcoming speech input. The priority or prioritization determines that a neutral short command causes the vehicle function sending the information to the speech dialogue system to be controlled; the other functions shown simultaneously by graphics on the display surface cannot be controlled with the short command. If the speech dialogue system confirms the priority, the function requesting speech input is marked so that the user can easily identify which function is reacting to their short command. In an advantageous way, the function reacting to a neutral short command only takes place when the speech dialogue system confirms the priority and marking takes place. This allows the speech dialogue system to give preference to other dialogues and also prevents, for example, the marking from taking place before the speech dialogue system is ready to receive input.
The operating device according to the invention now serves to control vehicle functions and comprises at least one microphone and one screen. It is set up to carry out the method in one of the described design variants and thus provides the hardware on the device side for carrying out the method in one of the embodiments according to the invention.
Further advantageous embodiments of the method according to the invention and of the operating device result from the exemplary embodiment which is described in more detail below with reference to the figures, in which:
In the representation of
System A now requests a speech dialogue from system B for a function displayed by means of a graphic on a display surface, such as a screen, i.e., the system A prepares system B to receive a speech input. System B starts this dialogue by being ready to receive and recognizes an incoming speech command of a person using a vehicle (not shown here). Upon receiving the speech command after a request to be ready to receive has been issued according to the arrow labelled 2, system B returns information corresponding to the speech command received to system A according to the arrow 3. System A can then start the vehicle function I, II, III assigned to the corresponding speech command.
The problem here is that the person using the operating device 1 does not know which vehicle function they are addressing when using a neutral short command such as “start”. Rather, they must give an unambiguous command in order to address a specific function, or it has to be accepted that all vehicle functions I, II, III reacting to the neutral short command are processed one after the other. This is not very convenient in practice.
In the representation in
In an additional, or alternative, embodiment it can be provided that a user can also use a short command to activate vehicle functions I, II, III displayed on a screen and represented by a graphic or button, without the above-mentioned request to the speech dialogue system, as long as this is marked as reacting to a short command. This variant can be implemented, for example, when the speech recognition system is preferably continuously ready to receive, i.e., is activated even without a keyword.
Since a function, prioritized for the input of a neutral short command, is marked as being distinctive from the other functions, a user then knows which of the vehicle functions I, II, III can be activated with a speech input of the neutral short command.
For this purpose, the operating device 1 comprises a screen 7, represented in
If the person using the operating device 1 wishes to address a different vehicle function, such as vehicle function II or III, they can do so at any time by either using an unambiguous command such as “start vehicle function II” or by moving the marking 4 from its previous position for vehicle function I to a position for another vehicle function II, III, for example by using a look or pointing gesture, in order to make this vehicle function II, III the vehicle function that can be executed directly by the short command 8.
This results in an operating device 1 that simply and efficiently enables convenient operation, even with neutral short commands 8, and therefore very quickly and intuitively.
Using the illustration in
The neutral short command 8 following the generation of the marking, which could also be described as an implicit command, therefore now prioritizes the vehicle function II, as shown by the marking 4, so that the incoming call is accepted when the speech command “start” is given, for example.
Case b) would be the other way around. Here, the system A prioritizes the vehicle function I, i.e., the switching on of seat heating, and the system B confirms this priority. Accordingly, the marking 4 is arranged in the region of the vehicle function I. In this case, the seat heating is switched on via the neutral short command “start”.
Case c) is now based on this case b). Here as well, the system A prioritizes the vehicle function I, i.e., the seat heating for example. Here however, the system B, i.e., the speech control, does not confirm this priority, but prioritizes the individual dialogue. In this case, a marking 4 of one of the vehicle functions I, II does not take place. Consequently, this means that none of the vehicle functions I, II shown on the display surface can be controlled via a neutral short command 8 or implicit speech command, as is obvious to the user. The neutral short command activates, for example, another vehicle function determined by system B and/or an explicit speech command is required here, i.e., an unambiguous command which explicitly names the desired function represented on the display surface by means of the graphic.
Although the invention has been illustrated and described in detail by way of preferred embodiments, the invention is not limited by the examples disclosed, and other variations can be derived from these by the person skilled in the art without leaving the scope of the invention. It is therefore clear that there is a plurality of possible variations. It is also clear that embodiments stated by way of example are only really examples that are not to be seen as limiting the scope, application possibilities or configuration of the invention in any way. In fact, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete manner, wherein, with the knowledge of the disclosed inventive concept, the person skilled in the art is able to undertake various changes, for example, with regard to the functioning or arrangement of individual elements stated in an exemplary embodiment without leaving the scope of the invention, which is defined by the claims and their legal equivalents, such as further explanations in the description.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2022 000 387.2 | Feb 2022 | DE | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2023/051811 | 1/25/2023 | WO |
