Patent Application
20220165261
The drive-thru experience is becoming more time consuming for most restaurants. The average time in the drive thru is now 255 seconds to fulfill an order which is up 60 secs from the previous year. This increase can be attributed to increased menu options for the customer to review, increased lines due to additional people ordering in vehicle and incorrect order delays once they arrive at the pickup window. This issue often leads to the abandonment of the line or the user choosing to walk into the restaurant instead. Its estimated that $178 million dollars per 2000 stores is loss at the drive thru window annually.
Additionally, the COVID19 pandemic has dramatically increased pickup and drive-thru orders, since many states have banned indoor dining in an effort to slow the spread of the virus. Restaurants are struggling to handle the volume of orders both from drive-thru orders and pickup orders. Restaurants were not equipped from staffing and technology standpoints to move their primary mode of business from indoor dining to drive-thru and pickup.
Given that some restaurants are no longer allowed to accepted diners or can only accept a reduced volume of diners, many restaurants are searching for ways to improve their ability to handle the volume associated with drive-thru and pickup orders. Furthermore, because restaurants have lost all or nearly all indoor diners due to the pandemic, restaurants are also simultaneously searching for ways to increase order volumes associated with drive-thru and pickup orders. Yet, increasing order volume is challenging for these restaurants when existing customer experiences associated with drive-thru orders and pickup orders were unfavorable to the restaurants even before the pandemic hit.
In various embodiments, methods and a system for voice-based pre-staged transaction processing are provided.
According to an embodiment, a method for voice-based pre-staged transaction processing is presented. A voice-based natural language session is established with a user who is operating a user device. An establishment and options selected for a pre-staged order with the establishment are determined during the voice-based natural language session. The pre-staged order is placed with the establishment on behalf of the user with the options. The establishment is instructed to begin preparing the pre-staged order for pickup by the user based on location data reported by the user device.
Furthermore, the various components (that are identified in the
As used herein and below, the terms “user,” “consumer,” “user,” and “customer” may be used interchangeably and synonymously. The terms refer to an individual placing an order at a transaction terminal.
As will be demonstrated more clearly herein and below, system 100 permits users to use natural-language and hands fee pre-stage ordering, where the voice orders are translated from audio to text and processed via Application Programming Interfaces (APIs) with the corresponding order systems of retailers. Further, when a location of the user is determined to be in-route to a pickup location, order fulfillment terminals are notified to begin preparing the pre-staged order to time the preparedness of the order with the arrival of the user at the pickup location. The system 100 is particular useful to users placing orders while in vehicles and traveling. The system 100 also allows retailers to efficiently process orders and fulfill orders to thereby reduce drive-thru times and increase timely order fulfillment.
System 100 includes a server/cloud 110, one or more user devices 120, one or more order servers 130, one or more navigation servers 140, and one or more fulfillment terminals 150.
Server/cloud 110 comprises at least one processor 111 and a non-transitory computer-readable storage medium 112. Medium 112 comprises executable instructions representing an order voice-based chatbot 113, order APIs 114, fulfillment terminal APIs 115, a navigation API 116, a pre-stage order manager 117, and a customer profile manager 118. Executable instructions when executed by processor 111 from medium 112 cause processor 111 to perform the processing discussed herein and below with respect to 113-118.
User device 120 comprises a processor and a non-transitory computer-readable storage medium. The medium comprising executable instructions for a mobile app (app) 121. The executable instructions when executed by the processor from the medium cause the processor to perform the processing discussed herein and below with respect to app 121.
Each order server 130 comprises at least one processor and non-transitory computer-readable storage medium. The medium comprising executable instructions for an order system 131. The executable instructions when executed by the processor from the medium cause the processor to perform the processing discussed herein and below with respect to order system 131.
Each navigation server 140 comprises at least one processor and non-transitory computer-readable storage medium. The medium comprising executable instructions for a navigation service 141. The executable instructions when executed by the processor from the medium cause the processor to perform the processing discussed herein and below with respect to navigation service 141.
Each fulfillment terminal 150 comprises at least one processor and non-transitory computer-readable storage medium. The medium comprising executable instructions for a fulfillment interface 151. The executable instructions when executed by the processor from the medium cause the processor to perform the processing discussed herein and below with respect to fulfillment interface 151.
During operation of system 100 voice-based pre-staged orders are processed on behalf of users/customers in manners and embodiments that are now discussed with reference to
Mobile app 121 presents a natural language based front-end user interface to the user and, optionally, a Graphical User Interface (GUI) for touch-based interaction and/or visual-based confirmation of a natural language voice-based session with server/cloud 110. Mobile app 121 when activated on user device 120 establishes a wireless connection to order voice-based chatbot 113. Mobile app 121 may also interact with location services on device 120 to continuously report location information for the device 120 and a device identifier for device 120 to chatbot 113 and/or pre-stage order manager 117.
Once a connection is made between app 121 and chatbot 113, a natural-language voice session is established. Chatbot 113 is configured to receive voice audio spoken by the user into a microphone associated with user device 120 and convert the audio into text comprising actions based on user intentions; the actions to be processed on behalf of the user. Similarly, Chatbot 113 is configured to receive as input text-based information during the natural language voice session from pre-stage order manager 117. The text-based information is translated by chatbot 113 and communicated to the user during the natural language voice-based session and spoken feedback that is played over a speaker associated with user device 120 to the user.
A user can initiate a connection and a corresponding session with chatbot 113 in a variety of manners. For example, app 121 may listen for a spoken wake-up word causing the session to be established. In another case, app 121 may establish a session with chatbot 113 based on user activation of a GUI option presented within a user-facing GUI when app 121 is initiated on device 120. In yet another situation, when user initiates app 121 on device 120 a session with chatbot 113 is automatically established by app 121 with chatbot 113 and the app 112 autonomously speaks to the user asking the user how can I help or would you like to place a pre-staged order?
Once the app 121 has a session with chatbot 113, the user can speak in natural language a desired order and/or a desired retailer that the user wants the order pre-staged with to chatbot 113. The user may also ask questions of chatbot 113, such as what places near me or in the direction that I am traveling have Chinese food or have cheese conies? Chatbot 113 determines the intention of the user, such as a question about a specific type of food, a specific restaurant, a specific distance to a nearby food establishment, etc. The intention can also be an instruction for a specific order, such as order me a double cheeseburger from McDonalds®.
Chatbot 113 determines actions that need to be processed based on the detected intentions from the user's voice statements during the session. The actions are then processed as operations with pre-stage order manager 117. Results returned by 151 are provides as text input information to chatbot 113. Chatbot 113 then translates the text input information and communicates back to the user through speech as a response to the user's initial question or instruction (based on the determined user intention).
Pre-stage order manager 117 manages all text-based actions determined to be necessary by chatbot 113 during the session with the user. Manager 117 interacts with a corresponding order system 131 using order API 114, a corresponding navigation service 141 using navigation API 116, and a corresponding fulfillment interface 151 using fulfillment API 116. Results of actions are returned by 131, 141, and/or 151 and provided by manager 117 for translation into natural spoken language by chatbot 113 for communication to the user during the session as feedback to the initial user's spoken intention.
Manager 117 may also interact with customer profile manager 118 to obtain a profile for a registered user/customer. The profile may be linked to a user device identifier for user device 120. The profile may include a transaction history for transactions of the user, username, user home address, a payment method (payment card and/or payment service), preferred restaurants, preferred food items, disfavored restaurants, disfavored food items, preferred price point for food purchases, etc.
Reported device location information for device 120 from app 121 permits manager 117 to identify where the user is located and even a direction of travel for the user (based on changing device locations for device 120). Moreover, the difference in time between two reported device locations, permits manager 117 to compute both a direction of travel for device 120 and a rate or travel or speed that device 120 is traveling. This allows manager 117 to know when the user is stationary or when the user is traveling in a vehicle. The user can also be determined to be at a home address (using the profile from profile manager 118) or can be determined to be traveling in the direction of the home address or away from the home address.
Manager 117 uses actions associated intentions directed to questions posed by the user as determined by chatbot 113 to locate specific restaurants or any restaurant that can provide the information to satisfy the request. This is done by manager 117 using the location information, speed of travel, and direction of travel to interact with navigation services 141 using navigation API 116 to obtain the specific restaurants or any restaurant within a predefined distance or within a distance that will be reached within a predefined amount of time (based on speed and direction of travel). The names, distances, and time to reach information can be provided as text input information to chatbot 113. Chatbot 113 translates to speech and communicates to the user via app 121. Additionally, specific menu items and prices for any given restaurant can be obtained by manager 117 using the corresponding order API 114 and interacting with the corresponding order system 131. Again, menu items and prices for the specific restaurants are provided as text input information to chatbot 113. Chatbot 113 translates to speech and communicates to the user via app 121. This interaction between the user (via app 121), chatbot 113, and manager 117 continues during the voice session as a dialogue with the user.
At some point during the dialogue (note this may be at the very beginning of the session), the user speaks an intention to place a specific order with a specific restaurant. Chatbot 113 provides the order details to manager 117. Manager 117 uses an order API 114 for a needed order system 131 and places a pre-staged order for pickup by the user with the corresponding restaurant using the order details. The order details may include a payment card, or a payment service obtained by manager 117 from a profile of the user via profile manager 118. The order details may also include an estimated or expected pickup time. Manager 117 may calculate the estimated pickup time based on the direction of travel and speed associated with the location data for device 120; alternatively, during the dialogue the user may have communicated the expected pickup time (the user may want to go somewhere else first or get gas for their vehicle before heading to pickup the order.
Manager 117 continues to monitor the order estimated and expected pickup time and location data of device 120 once the order is placed on behalf of the customer. When manager 117 determines that the food preparation time (based on historical data or data provided by the restaurant for orders) will be completed and substantially coincide with the arrival time of the user (based on the location data), manager 117 uses a fulfillment terminal API 115 and sends a message to the corresponding restaurant's fulfillment interface 151 with the order number and an instruction to begin food preparation of the order now as the user/customer is expected to arrive within X minutes. Manager 117 may also send a message to chatbot 113 to communicate to the user that the order is being prepared for pickup by the restaurant along with any pickup location details provided by fulfillment interface to manager 117 during their interaction. For example, the restaurant may have instructions to pickup the order in a predesignated area of its packing lot, which is not associated with any drive-thru and which is not associated with the user leaving the car to come into the restaurant. This allows the restaurant to manage pre-staged orders for pickup separately from its drive-thru customers and separate from customers dining in the restaurant (assuming this is even permitted during COVID19).
An example, process flow utilizing the voice-based pre-staged transaction processing of system 100 may proceed as follows. It is noted that this example process flow is intended to be illustrative and non-limiting as a variety of other voice sessions and voice dialogues associated with other process flows are foreseeable by system 100.
A consumer/user initiates the voice interaction within their vehicle utilizing user device 120 and app 121 to create a voice session with chatbot 113 associated with a voice dialogue with the consumer.
The consumer requests a food or restaurant choice they are interested in from the chatbot 113. Chatbot 113 interacts with manager 117. Manager 117 analyzes the text actions translated by chatbot 113 for the request, analyzes location data returned by app 121 for the request, and interacts with navigation service 141 using API 116 and order systems 131 using API 114. Manager 117 determines specific menu items satisfy the request from a specific restaurant and provides a text feedback information to chatbot 113. Chatbot 113 translates the menu items to speech and communicates to the consumer via app 121 during the session and dialogue. The consumer responds with specific options via voice to chatbot 113. Chatbot 113 provides the options as translated text input information to manager 117. The options are communicated to the proper order system 131 as a pre-staged consumer order by manager 117 and confirmed. The confirmation is sent from manager 117 to chatbot 113 and communicated to the consumer during the voice session and dialogue. Payment information may also be provided by manager 117 for the pre-staged consumer order to order system 131 based on the consumer's profile or based on specific voice-based payment card information provided by the consumer to chatbot 113 during the voice session and voice dialogue.
Manager 117 continues to monitor location data for device 120 and the corresponding pre-staged consumer order. When manager 117 determines that the device 120 is within a predefined range or time for arriving at the restaurant, manager 117 sends a message to the appropriate fulfillment interface 151 stating the order should be prepared now. Any pickup instructions are provided from interface 151 to manager 117, manager 117 communicates to chatbot 113, and chatbot 113 provides the pickup instructions to the consumer as voice during the voice session and voice dialogue. The consumer drives on site and picks up food from the appropriate drive thru area for pre-staged orders or other area defined by the pickup instructions.
Manager 117 detects from the location data of device 120 that the consumer has arrived at the restaurant and sends another message to fulfillment interface 151 informing staff that the consumer associated with the order is onsite for pickup of the order.
In an embodiment, chatbot 113 may presented to the consumer on a display associated with user-device 120 as an animation or an avatar that is synchronized with the auto-generated speech communicated by chatbot 113.
In an embodiment, chatbot 113 is configured to use a spoken language and dialect associated with the user. In an embodiment, during voice training, chatbot 113 detects the user's spoken language and dialect. In an embodiment, the profile for the user includes spoken language identifiers and dialect identifiers.
In an embodiment, the menu items presented to a user either visually or through auto-generated speech is determined based on the user's history of ordering and based on trends associated with ordering from establishments within a zone or region where the user is ordering.
In an embodiment, user device 120 is a phone, a tablet, a laptop, a built-in vehicle computing device (electric or non-electric vehicle with built in computing device), or a wearable processing device.
In an embodiment, the fulfilment terminal is a backend kitchen- based ordering terminal or monitor used by staff to prepare orders within a given restaurant.
In an embodiment, system 100 is processed for pre-staging an order for pickup that is not associated with food take out, such a groceries, or non-food products.
In an embodiment, app 121 translates user spoken voice into text and provides over the established connection to chatbot 113 and chatbot provides feedback as text back to app 121. In this way, the bandwidth associated with audio being transmitted during the voice session can be substantially reduced.
In an embodiment, app 121 is provided as a stand-alone mobile device app, a vehicle system-based app, a browser-based app, or an app integrated into a social media system (Facebook®, Instagram®, Twitter®, etc.).
In an embodiment, 113-118 is provided as an enhancement to an existing voice-bases service, such as Amazon® Echo®, Google® Home®, Apple® Siri®, etc.
These and other embodiments will now be discussed with reference to
In an embodiment, the device that executes the pre-staged ordering chatbot is server 110. In an embodiment, the server 110 is a cloud-based processing environment comprising a collection of physical servers cooperating as a single logical server (a cloud 110).
In an embodiment, the pre-staged ordering chatbot is all or some combination of the chatbot 113, order APIs 114, fulfillment terminal APIs 115, navigation API 116, pre-stage order manager 117, and/or customer profile manager 118.
At 210, pre-staged ordering chatbot establishes a voice-based natural language session (session) with a user operating a user device.
At 220, the pre-staged ordering chatbot determines that an establishment and options are selected by the user for a pre-staged order with the establishment during the session.
In an embodiment, at 221, the pre-staged ordering chatbot locates available establishments and available options based on a direction of travel associated with the user device from the location data. The pre-staged ordering chatbot can calculate the direction of travel based on changes in locations reported by the user device. The pre-staged ordering chatbot communicates the available establishments and the available options to the user through speech during the session for user voice selection.
In an embodiment of 221 and at 222, the pre-staged ordering chatbot filters the available establishments and the available options based on a profile associated with the user.
In an embodiment of 222 and at 223, the pre-staged ordering chatbot locates the available establishments and the available options based on the direction of travel and the rate of travel for the user device calculated from the location data (e.g., rate or speed can be calculated based on time of a first location and time of a second location).
In an embodiment, at 224, the pre-staged ordering chatbot locates available establishments and available options based on a desired region (city, mile marker, distance ahead of the user, zip code, etc.) communicated by the user during the session. The pre-staged ordering chatbot communicates the available establishments and the available options to the user through speech during the session for user selection.
In an embodiment, at 225, the pre-staged ordering chatbot locates available establishments and available options based on a desired time for pickup of the pre-staged order that is communicated by the user via speech during the session. The pre-staged ordering chatbot communicates the available establishments and the available options to the user through speech during the session for user selection.
In an embodiment, at 226, the pre-staged ordering chatbot locates available establishments and available options based on the location data reported by the user device. The pre-staged ordering chatbot provides the available establishments and the available options to a Graphical User Interface (GUI) for touch selection by the user on the user device during the session. Additionally, the pre-staged ordering chatbot simultaneously communicates the available establishments and available options to the user through speech during the session for an additional mechanism by which the user can make selections of the establishment and the corresponding options.
At 230, the pre-staged ordering chatbot places the pre-staged order with the establishment on behalf of the user with the options.
In an embodiment, at 231, the pre-staged ordering chatbot translates spoken voice of the user associates with the establishment and the options to text and identifies the establishment and the options from the text. The pre-staged ordering chatbot processes an API associated with an order system of the establishment to place the pre-staged order with the options on behalf of the user.
In an embodiment, at 232, the pre-staged ordering chatbot provides a payment card or a payment service to the establishment for payment of the pre-staged order based on a profile associated with the user.
In an embodiment, at 233, the pre-staged ordering chatbot obtains payment details from the user during the session through speech provided by the user for the payment details. The pre-staged ordering chatbot provides the payment details to the establishment as payment for the pre-staged order.
At 240, the pre-staged ordering chatbot instructs the establishment to begin preparing the pre-staged order for pickup by the user based on location data reported by the user device.
In an embodiment, at 241, the pre-staged ordering chatbot estimates an order preparation time required by the establishment for the pre-staged order. The pre-staged ordering chatbot determines a time required for the user device at a current location to reach an establishment location is equal to or is less than the order preparation time by a threshold amount of time. When this is determined, the pre-staged ordering chatbot instructs a fulfillment terminal at the establishment location to being preparing the pre-staged order on behalf of the user.
In an embodiment, at 250, the pre-staged ordering chatbot informs the establishment that the user has arrived at the establishment or a designated establishment location for pickup of the pre-staged order based on location data reported by the user device.
In an embodiment, the device that execute the speech and location-based pre-staged order manager is server 110. In an embodiment, the server 110 is a cloud processing environment (cloud 110).
In an embodiment, the speech and location-based pre-staged order manager is all of, or some combination of: chatbot 113, order APIs 114, fulfillment terminal APIs 115, navigation API 116, pre-stage order manager 117, customer profile manager 118, and/or the method 300.
The speech and location-based pre-staged order manager presents another and, in some ways, enhanced processing perspective of the method 200.
At 310, the speech and location-based pre-staged order manager engages a consumer in a natural-language voice dialogue for placing a pre-staged order with a restaurant while the consumer is driving a vehicle with a consumer device in possession of the consumer or integrated into the vehicle.
In an embodiment, at 311, the speech and location-based pre-staged order manager provides available restaurants for voice selection by the consumer based on a direction of travel of the vehicle, a region provided by the consumer, or a profile of the consumer.
At 320, the speech and location-based pre-staged order manager obtains menu options from the consumer for the restaurant during the voice dialogue.
In an embodiment of 311 and 320, at 321, the speech and location-based pre-staged order manager provides available menu options for voice selection by the consumer based on the restaurant selected by the consumer.
In an embodiment of 321 and at 322, the speech and location-based pre-staged order manager simultaneously provides the available menu options for touch selection by the consumer within a GUI of the consumer device.
At 330, the speech and location-based pre-staged order manager processes an API and places the pre-staged order with an order system associated with the restaurant using the menu options.
In an embodiment of 322 and 330, at 331, the speech and location-based pre-staged order manager provides payment details associated with the profile of the consumer for payment of the pre-staged order to the order system.
At 340, the speech and location-based pre-staged order manager determines that the consumer device will arrive at the restaurant within a period of time that is equal to an order preparation time for preparing the pre-staged order based on location data associated with the consumer device and a restaurant location for the restaurant.
In an embodiment of 331 and 340, at 341, the speech and location-based pre-staged order manager estimates the order preparation time based on the restaurant, a time of day, a day of week, and pending unfulfilled orders associated with the retailer.
At 350, the speech and location-based pre-staged order manager sends a message to a fulfillment terminal of the retailer to begin preparing the pre-staged order to time arrival of the consumer at the restaurant location to substantially coincide with completion of the pre-staged order based on 340.
It should be appreciated that where software is described in a particular form (such as a component or module) this is merely to aid understanding and is not intended to limit how software that implements those functions may be architected or structured. For example, modules are illustrated as separate modules, but may be implemented as homogenous code, as individual components, some, but not all of these modules may be combined, or the functions may be implemented in software structured in any other convenient manner.
Furthermore, although the software modules are illustrated as executing on one piece of hardware, the software may be distributed over multiple processors or in any other convenient manner.
The above description is illustrative, and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. The scope of embodiments should therefore be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
In the foregoing description of the embodiments, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting that the claimed embodiments have more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Description of the Embodiments, with each claim standing on its own as a separate exemplary embodiment.
