Patent Application
20240029724
The present disclosure relates to apparatus and methods for use with a voice assistant. Aspects of the invention relate to a control system for a voice assistant system, to a system, to a method and to computer software.
It is known to provide a voice assistant or digital personal assistant (DPA) which are functionality built into either a dedicated device or hosted on a computing device which allows a user to interact with the device using spoken commands. For example, the user may speak a name of the voice assistant or DPA, or a customised wake-up word, to activate or wake up the voice assistant followed by a command or question. For example, the user may speak ‘Sebastian, what is the weather like now?” where Sebastian is a name assigned to the voice assistant which is followed by the command in the form of a question about the weather, as an example. The device implementing the voice assistant communicates an indication of the spoken commands, such as audio data containing the spoken commands, to a server or cloud computing system associated with the voice assistant for analysis and acting upon, such as by formulating an audible response thereto. The response is returned to the device and output as an audible response. The server of the voice assistant may have a set of functionalities for acting upon commands or answering questions.
It is possible for third party developers to expand a functionality of a voice assistant by providing additional applications which interact with the voice assistant system. For example. An additional application may be hosted on the same or a different server or cloud computer to that of the voice assistant. The additional application defines a set of commands which may be accessed by the voice assistant. The application receives an indication of commands and data from the voice assistant to provide the additional functionality and returns appropriate data in response, which the voice assistant uses to form an audible response. In some instances, the additional application may be associated with a vehicle, such as provided by a vehicle manufacturer. The additional application may provide a plurality of actions or functions which may each be invoked by the user using appropriate voice commands, an indication of which is passed to a device or computer hosting the additional application. One such function may be provided to determine whether a user's vehicle is locked. The additional application receives an indication from the voice assistant querying the lock status of the user's vehicle. The additional application communicates with the vehicle to determine the lock status and a corresponding indication is returned from the additional application to the voice assistant. The voice assistant may output a response, or respond, to the user with a spoken response indicating, for example, that the vehicle is locked.
It is desired to make use of such additional functions more convenient for users of the voice assistant system. It is an aim of the present invention to address one or more of the disadvantages associated with the prior art.
Aspects and embodiments of the invention provide a control system, a system, a method and computer software as claimed in the appended claims
According to an aspect of the present invention there is provided a control system for a voice assistant system, the control system comprising one or more controllers, the control system comprising an interface configured to receive a request from the voice assistant system, wherein the request is indicative of a task requested by a user of the voice assistant system, and processing means arranged to execute, in dependence on the request, a task handler corresponding to the task, wherein the task handler is arranged to cause the control system to perform a plurality of actions associated with the task, to generate a response in dependence on an output of at least some of the plurality of actions and to output the response via the interface to the voice assistant system. Advantageously by invoking the task handler to perform the plurality of actions, each action does not need to be separately invoked. Advantageously reduced communication is required to perform the plurality of actions. Advantageously the response combines an output of the plurality of actions which may thereby improve efficiency. Advantageously the user does not need to use the voice assistant system to request each of the plurality of actions.
According to another aspect of the invention, there is provided a control system for a voice assistant system for implementing actions associated with a vehicle, the control system comprising one or more controllers, the control system comprising an interface configured to receive a request from the voice assistant system, wherein the request is indicative of a task requested by a user of the voice assistant system, and processing means arranged to execute, in dependence on the request, a task handler corresponding to the task, wherein the task handler is arranged to cause the control system to perform a plurality of actions associated with the task, to generate a response in dependence on an output of at least some of the plurality of actions and to output the response via the interface to the voice assistant system. Advantageously by invoking the task handler to perform the plurality of actions, each action does not need to be separately invoked. Advantageously reduced communication is required to perform the plurality of actions. Advantageously the response combines an output of the plurality of actions which may thereby improve efficiency.
Each of the actions may be independently invokable by the voice assistant system. Advantageously each action may be independently requested, or the task may be requested thereby providing flexibility.
The processing means may be arranged to communicate with the vehicle to perform at least some of the plurality of actions. Advantageously at least some of the actions may interact with the vehicle.
Optionally at least some of the plurality of actions each correspond to a request for data from the vehicle. Advantageously data may be obtained from the vehicle, which may be indicative of a state of the vehicle. The processing means may be arranged to communicate with the vehicle to obtain the data and to generate the response in dependence on the data.
Advantageously the voice assistant system is provided with the response in dependence on the vehicle data.
A first action optionally comprises obtaining data indicative of a status of the vehicle. Advantageously the voice assistant system is provided with an indication of the status.
A second action optionally comprises obtaining data indicative of a calculated range of the vehicle. Advantageously the voice assistant system is provided with an indication of the range of the vehicle.
A third action optionally comprises activating a security mode of the vehicle. The security mode may be a high-security mode of the vehicle. Advantageously the security mode of the vehicle can be configured using the voice assistant system.
At least some of the plurality of actions each correspond to a request for the vehicle to perform a respective function. Advantageously the voice assistant system may be used to cause the vehicle to perform those functions. The processing means may be arranged to communicate with the vehicle to cause the vehicle to perform the respective functions.
The processing means may be arranged to receive data indicative of the function being performed by the vehicle and to generate the response in dependence thereon.
Advantageously the voice assistant system is provided with an indication that the function has been performed. The voice assistant may provide an audible indication in dependence thereon.
Optionally one or more of the functions comprises an instruction to cause the vehicle to actuate an aperture of the vehicle. Advantageously the aperture may be actuated via the voice assistant system.
The actuation may be to close or open the aperture. The aperture may be a door or window of the vehicle. Advantageously the door or window may be actuated via the voice assistant system.
According to yet another aspect of the invention, there is provided a system, comprising a control system according to any preceding claim, and a server for a voice assistant system arranged to provide the request to the control system indicative of the task requested by the user and to receive the response from the control system.
The server may be arranged to communicate the request to the control system according to a predetermined routine.
According to another aspect of the invention, there is provided a computer-implemented method for a voice assistant system for implementing actions associated with a vehicle, comprising receiving a request from a voice assistant system, wherein the request is indicative of a task requested by a user of the voice assistant system, executing, in dependence on the request, a task handler corresponding to the task, wherein the task handler is arranged to cause performance of a plurality of actions associated with the task, to generate a response in dependence on an output of at least some of the plurality of actions, and outputting the response to the voice assistant system.
The method may comprise communicating with the vehicle to perform at least some of the plurality of actions.
At least some of the plurality of actions may each correspond to a request for data from the vehicle. The method optionally comprises communicating with the vehicle to obtain the data and generating the response in dependence on the data.
A first action optionally comprises obtaining data indicative of a status of the vehicle. A second action optionally comprises obtaining data indicative of a calculated range of the vehicle
At least some of the plurality of actions each correspond to a request for the vehicle to perform a respective function. The method optionally comprises communicating with the vehicle to cause the vehicle to perform the respective functions.
The method optionally comprises receiving data indicative of the function being performed by the vehicle and to generating the response in dependence thereon.
One or more of the functions comprises an instruction to cause the vehicle to actuate an aperture of the vehicle.
According to a still further aspect of the invention, there is provided a control system associated with a voice assistant system, the control system comprising one or more controllers, the control system comprising processing means arranged to determine a plurality of actions associated with a vehicle performed within a predetermined period of time in response to a request from a voice assistant system, generate a task handler corresponding to a task, wherein the task handler is arranged to cause a control system to perform the determined plurality of actions in response to a request for the task received from the voice assistant system, to generate a response in dependence on an output of at least some of the plurality of actions and to output the response to the voice assistant system. Advantageously the plurality of actions requested in temporal proximity are identified and a task handler generated to allow performance of those actions without a requirement to be separately invoked.
The generating the task handler optionally comprises defining an interface for the task handler to receive data associated with each of the plurality of actions. Advantageously the task handler is configured to receive data appropriate for those actions.
According to yet another aspect of the invention, there is provided a method associated with a voice assistant system, the method comprising determining a plurality of actions associated with a vehicle performed within a predetermined period of time in response to a request from a voice assistant system, generating a task handler corresponding to a task, wherein the task handler is arranged to cause a control system to perform the determined plurality of actions in response to a request for the task received from the voice assistant system, to generate a response in dependence on an output of at least some of the plurality of actions and to output the response to the voice assistant system.
According to another aspect of the invention, there is provided computer readable instructions which, when executed by a computer, are arranged to perform a method as described above. The computer readable instructions may be tangibly stored on a computer readable medium. The computer readable medium may be non-transitory. The computer readable instructions may comprise computer software.
Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and/or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and/or features of any embodiment can be combined in any way and/or combination, unless such features are incompatible. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and/or incorporate any feature of any other claim although not originally claimed in that manner.
One or more embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:
With reference to
The controller 110, and thus the control system 100, comprises an input means and an output means. The input means 140 may comprise an electrical input of the controller 110. The output means may comprise an electrical output of the controller 110. In the embodiment illustrated in
The voice assistant device 210 comprises at least one processor or processing device 215, a microphone 220 for generating audio data and a speaker 230 for converting an audio signal into audible sound. The voice assistant device 210 also comprises a memory storing computer-readable instructions which are executed by the processor 215 and a communication module for communicating with the voice assistant server 250, such as via the internet using wired, wireless, or a combination thereof, communication.
The voice assistant server 250 is communicably coupled with the control system 100. In one embodiment, the control system 100 and the voice assistant server 250 may share one or more computing resources i.e. may be implemented on the same computer system, or may be implemented on separate computer systems which communicate via one or more networks such as over the internet.
The control system 100 is associated with a vehicle 240. The control system 100 is provided for implementing actions associated with the vehicle 240. An action may be a request for information associated with the vehicle 240, such as an indication of a status of the vehicle 240 or one or more of its sub-systems, or the action may be a request to control the vehicle 240 or one or more of its sub-systems. For example, the action may comprise a request for an indication of a range of the vehicle 240 using its current fuel load, such as a battery charge level of the vehicle 240, or the action may be a request to close a window or to lock the vehicle 240. Other actions may be envisaged. The vehicle 240 is associated with a user of the voice assistant device 210. In use, the system 200 may comprise a plurality of voice assistant devices 210 each associated with a respective one of a plurality of vehicles 240. The user of each voice assistant device is able to perform actions with respect to their specific vehicle 240. However, for clarity one vehicle 240 and one voice assistant device 210 are described.
The control system 100 is arranged to have registered with the voice assistant server 250, such that the voice assistant server 250 is aware of the presence of the control system 100. For example, the voice assistant server is aware of an identification of the control system 100, such as a name of the control system 100. The voice assistant server 250 may associate the control system with a name ‘Jaguar Land Rover’ as an example. The voice assistant server 250 is aware of one or more actions provided by the control system 100 for the voice assistant server 250. For example, the control system 100 may provide an action ‘range’ which in response to a request for ‘range’ returns a numeric value indicative of the remaining range of the vehicle 240. Similarly, the control system 100 may provide an action ‘lock’ for locking the doors of the vehicle 240 which, in response to a corresponding request, returns a binary value indicative of the success of the locking of the vehicle doors. It will be realised that other actions may be provided by the control system 100 to the voice assistant server 250.
The voice assistant device 210 listens using its microphone 220 for audio representing a Wake up Word (WuW), such as ‘Sebastian’ i.e. a name of the voice assistant, which is followed by a spoken command corresponding to an action. For example, a combination of WuW and action may be ‘Sebastian, what is the range of my vehicle?’ The processor 215 performs an initial analysis of audio data provided by the microphone 220 corresponding to and following the WuW to make an initial confirmation of recognition of the WuW in the audio data. If the initial confirmation is positive, then the audio data is communicated 230 to the voice assistant server 250. The voice assistant server 250 is arranged to analyse the audio data to confirm the presence of the WuW in the audio data. Upon confirming the presence of the WuW, the voice assistant server 250 is arranged to perform speech recognition on the audio data following the WuW in order to determine the command present in the audio data following the WuW. In the example, the command corresponds to ‘what is the range of my vehicle?’.
The voice assistant server 250, being aware of the actions provided by the control system 100, analyses the speech contained within the audio data to identify one or more actions requested by the user. For example, the voice assistant server 250 recognises that the speech ‘what is the range of my vehicle?’ corresponds to a request for the first action 310 ‘range’ provided by the control system 100. The voice assistant server 250 then communicates 260 a request for the action to the control system 100.
However, it has been recognised that, frequently, users of such voice assistant systems request a plurality of actions in close temporal proximity. For example, it has been observed that a user typically requests an indication of the remaining range of their vehicle 240, requests to lock the doors of their vehicle 240, and requests activation of a high-security setting of their vehicle 240 e.g. in an evening such as when they have finished using their vehicle 240 for the day. It will be appreciated that these are example actions and that other actions may be requested. Requesting each of these actions individually is inconvenient for the user and, furthermore, additional communication is required between the voice assistant server 250 and the control system 100 to separately request each action and to communicate a response of the control system 100 to the voice assistant server 250 for each action. Embodiments of the present invention aim to address one or more of these, or other, disadvantages.
Reference will also be made to
The method 400 starts at block 405 and comprises a block 410 of receiving a request 505 at the control system 100 from the voice assistant server 250. The request 505 is communicated 260 to the control system 100, such as via one or more data networks e.g. computer and/or telecommunications networks, for example over the Internet. In some embodiments where the control system 100 is functionally integral or co-located with the voice assistant server 250 the request 505 may be communicated via a memory or a local network to a processor 120 of the control system 100. The request 505 may comprise identifying information indicating a name of an action or task requested by the voice assistant server 250 i.e. corresponding to the user's spoken voice command. The request 505 may comprise data associated with the request, such as data identifying a user from whom the request originated i.e. user of the voice assistant device 210 and/or identifying the vehicle 240. The request 505 may identify a task associated with the request 505. As described above, the control system 500 may, for example, support three independently invokable actions 520, 530, 540 i.e. which may each be separately requested and provide a separate response. An action may correspond to an intent supported by the Amazon® Alexa® voice assistant system, for example. A task corresponds to a request for a plurality of actions, i.e. more than one of the actions 520, 530, 540, to be performed by the control system 500 and a combined response to be returned to the voice assistant server 250.
In block 420 it is determined whether the received request 505 is a request for an action supported by the control system 500 or a task corresponding to the task handler 510 supported by the control system 500, such as shown in
If the request 505 corresponds to one of the actions in block 420, the method 400 follows 425 to block 430 where a corresponding one of the actions 520, 530, 540 is invoked. If, however, the request 505 corresponds to the task handler 510, or one of the task handlers provided by the control system 500, the method 400 moves to block 450 wherein the corresponding task handler 510 is invoked or executed. In this example, the ‘Goodnight’ task handler 510 is invoked in block 450 corresponding to the received request 505.
The, or each, task handler 510 defines a plurality of actions 520, 530, 540 to be performed as part of the task. In the described example, the ‘Goodnight’ task defines all three of the actions 520, 530, 540, or intents, provided by the control system 500. That is, invoking the task handler 510 for the ‘Goodnight’ task causes all three of the actions 520, 530, 540 to be performed by the control system 500. Thus, the request 505 may provide any data necessary for all three of the actions 520, 530, 540 to be executed in dependence on the task handler 510. That is, the request indicative of the ‘Goodnight’ task may comprise data associated with each of the plurality of actions 520, 530, 540 to be performed as part of the task. In this sense, the request for the ‘Goodnight’ task is a combined request 505 which comprises data necessary to perform all of the plurality of actions 510, 520, 530 with which it is associated.
The task handler 510 defines which data of the request 505 is provided to each action of the plurality of actions 520, 530, 540. For example, a first part of data of the request 505 is provided to the first action 520 and a second part of the data of the request 505 is provided to the second action 530, etc.
As an example, a first action 510 requests an indication of the remaining range of the vehicle 240, a second action 520 requests to lock the doors of the vehicle 240, and a third action causes activation of a high-security setting of their vehicle 240.
In block 460 each of the actions 520, 530, 540 defined by the task handler 510 are invoked or executed as requested by the task handler 510. For example, each of the three actions 520, 530, 540 in this example may be sequentially invoked by the control system 500. The task handler 510 may define an order or execution of each action and/or whether at least some of the plurality of actions 520, 530, 540 may be performed in parallel. Each of the actions 520, 530, 540 may cause the control system 500 to communicate 270 with the vehicle 240, for example to request data or to request the vehicle 240 to perform a respective function i.e. to carry out an activity at the vehicle 240, such as to change a state or configuration or one or more systems of the vehicle 240. For example, one or more functions may comprise an instruction to cause the vehicle 240 to actuate an aperture or opening of the vehicle 240. The instruction may be to close or open an identified aperture of the vehicle 240. The aperture may be a door or window of the vehicle 240. That is, one or more of the actions may cause e.g. the windows of the vehicle 240 to close in order to secure the vehicle 240.
In order to advantageously reduce communication with the voice assistant server 250, where more than one of the actions 520, 530, 540 returns a response, such as an indication of the range of the vehicle or confirmation that the doors are locked and the high-security setting is activated, the task handler 510 is configured to collate the outputs of each of the actions 520, 530, 540 to form a combined response in block 470. That is, the task handler 510 receives the output of each of the actions 520, 530, 540 and assembles the combined response including data from each of the plurality of actions 520, 530, 540. The task handler may define a structure of the response, such that the voice assistant server 250 is informed that e.g. first data in the response corresponds to the range, second data is confirmation of the vehicle 240 being locked etc.
In block 480 the combined response is communicated 260 to the voice assistant server 250. At the voice assistant server 250 a response to the voice assistant device 210 is generated, such that at the voice assistant device 210 an audible response is output in dependence thereon. The audible output may, for example, provide an audible output conveying the range of the vehicle 240 provided as part of the combined response from the control system 500, an indication that the doors of the vehicle are locked and that the high-security setting is activated. For example, the voice assistant device 210 may audible output ‘The remaining range of your Jaguar is 210 km, the doors are locked and the vehicle is in a high-security mode’. Thus as can be appreciated, the user has been able to make a single audible request such as ‘Sebastian, Goodnight to my Jaguar’ which invokes the task handler 510 upon the control system 100, 500 to correspondingly invoke each of the plurality of actions 520, 530, 540.
As described above in relation to
In embodiments of the invention, the control system 100 is arranged to store data indicative of an invocation time of each action 520, 530, 540. That is, “invocation time” is understood to mean a time at which each action 520, 530, 540 is invoked by each respective user of a voice control device 210, rather than a duration of invocation.
The method 600 may be performed by the processor 120 of the control system 100 in some embodiments. It will be appreciated, however, that the method 600 may be performed by the processor of another computing device utilising the data indicative of the invocation time of each action 310, 320, 330 produced by the control system 100.
The method 600 begins at block 605. In block 610 a plurality of actions, or intents in some embodiments, which are invoked within a predetermined period of time 710 are identified. Referring to
Thus in block 610 a plurality of actions 520, 530 are selected by the control system 100 in dependence on those actions being historically performed in temporal proximity.
In block 620 a task handler 510 is generated. The task handler is generated such that, when invoked by a received request for the task handler, the task handler is configured to invoke the plurality of actions 520, 530 in dependence on the request. The task hander 510 is associated with a respective task, such as ‘Goodnight’ as described above, and is configured to perform the plurality of actions 520, 530 associated with the task, to generate a response in dependence on an output of at least some of the plurality of actions 510, 520 and to output the response to the voice assistant system 200.
Generating the task handler 510 in block 620 comprises defining an interface for the task handler to receive data associated with each of the plurality of actions 520, 530, 540. The interface is configured to receive data to perform each of the plurality of actions. The interface may also receive data returned from each of the plurality of actions, such that the task handler 510 can generate a response to the voice assistant system 200 in dependence on an output of at least some of the plurality of actions. The task handler 510 outputs the response to the voice assistant system 200 as a combined response i.e. comprising a response of each of the plurality of actions 520, 530, 540.
Whilst embodiments of the invention have been described with reference to actions associated with a vehicle, it will be appreciated that embodiments of the invention may be envisaged which are not associated with a vehicle. For example, embodiments of the invention may be associated with, for example, an IoT system or an automated home control system although it will be appreciated that this is not limiting. As an example, where the control system 100, 500 is associated with a home control system a first action may be arranged to actuate, such as to close, a first window of a home associated with the control system, a second action may be arranged to activate or turn on one or more lights of the home and a third action may be arranged to activate a home appliance. A corresponding task handler may be configured to perform the second and third actions in response to a request from a voice assistant system.
Thus, it can be appreciated that embodiments of the present invention are not restricted to use with vehicles.
It will be appreciated that various changes and modifications can be made to the present invention without departing from the scope of the present application.
The following numbered clauses define various further aspects and features of the present technique:
