Patent Application
20240144401
Alternative means of transportation are becoming more prevalent. Scooters, e-bikes, and other personal electric vehicles (PEVs) are the main form of transportation for many. Restaurant drive-thrus do not typically accommodate this type of vehicle and lobbies are often closed during later store hours such that the only availability for customers is through car drive-throughs.
A significant portion of the population lives in urban areas and may not even own a car. For safety reasons, drive-throughs typically prohibit walk through customers, wheelchair customers, and customers on bikes. During COVID restaurants were only allowing access to customers via their drive-throughs. This presented a significant problem for truckers who were unable to find food because their trucks could not fit through a drive-through, walk-in service was closed to the public, and the drive-throughs did not permit walkup customers for safety reasons.
In various embodiments, methods and a system for pickup order processing are presented. Images of vehicles and customers are captured in drive-through lanes. A check is made to ensure the customers are in a proper lane and if not, they are instructed to move to a proper lane to place an order. The images are further processed to provide a written description of the vehicle, if any, and the customer. The written description is integrated into the order workflow for the customer order for verifying the customer when the order is picked up at a designated pickup window.
Traditional drive-thrus include metal sensors, which are activated by motor vehicles but are not activated by bicycles and other PEVs. When the sensors are not activated, staff in the restaurant are unaware that a customer is at the drive-through desiring to place an order. As a result, the customer goes to the pay window or order fulfillment window only to turned away from services because of the restaurant's safety policies.
When the sensors are activated, assuming the drive-through includes a camera, an image of the vehicle is taken and associated with the vehicle and the order taken within the order system of the restaurant. This allows staff of the restaurant to properly associate an order with the customer in the vehicle because some fast-food restaurants can include multiple order taking drive-throughs, which do not always operate one after the other especially when an order in one of the lanes takes longer to give than an order in the other lane. To avoid charging the customers for the wrong orders and giving customers incorrect orders, the order number is usually linked to an image of the vehicle so staff at the payment window collect the correct fee for the correct order and staff at the pickup window gives the correct order to the correct customer.
Although governments and organizations are attempting to improve the climate by phasing out carbon emissions associated with vehicles that burn fossil fuels, there has been little incentive to use technology to accommodate individuals whose primary means of transportation is walking or via PEVs. A sizable population does not own a gas-powered car nor an electric car. Consumers are becoming more climate conscious and are using bikes, e-bikes, scooters, and other PEVs in larger numbers. Restaurants are potentially losing substantial revenue from this population, which was especially the case during the pandemic since truck drivers and other walk-up customers were unable to obtain in-store service.
These issues are alleviated with the teachings presented herein and below. Images are taken of drive-throughs when cars, walkup, and/or PEVs are present. The images are provided to a cloud-based order service where a trained machine-learning model (MLM) processes the image as input and produces as output the type of vehicle, if any, and a written description of the vehicle and/or the individual present at the drive-through or on the PEV. The output from the MLM can be used within a drive-through and order workflow for purposes of identifying when customers are in an improper drive-through lane of a restaurant so as to direct the customers to the proper lane and for purposes of providing descriptive written information to identify the customers and optionally their vehicles. The descriptive written information is then linked and available with the order to staff that is responsible for fulfilling the order within the restaurant.
With the teachings provided herein, restaurants can provide two separate drive-through lanes; one for cars and one for walkups bicycles, scooters, and PEVs. The actual pickup area for the walkups, bicycles, scooters, and PEVs may be through the lane that the customers ordered from or available within the store.
As used herein, a “vehicle” includes a car, a bicycle/bike, a scooter, or other PEVs. A “non-car customer” includes an individual walker, a customer on a bike, a customer on a scooter, or a customer on any PEV.
Moreover, various components are implemented as one or more software modules, which reside in non-transitory storage and/or hardware memory as executable instructions that when executed by one or more hardware processors perform the processing discussed herein and below.
System 100 includes a cloud 110 or a server 110 (hereinafter referred to as “cloud 110”), one or more restaurant servers 120, one or more point-of-sale (POS) terminals or servers 130, drive-throughs 140, and cameras 150. Cloud 110 includes at least one processor 111 and a non-transitory computer-readable storage medium (hereinafter “medium”) 112, which includes executable instructions for an order service 113 and a machine-learning model (MLM) 114. The instructions when provided to processor 111 cause processor 111 to perform operations discussed herein and below for 113-114.
Each restaurant server 120 includes at least one processor 121 and medium 122, which includes executable instructions for an order system 123. The instructions when provided to processor 121 cause processor 121 to perform operations discussed herein and below for 123.
Each POS 130 comprises one or more processors 131 and medium 132, which includes executable instructions for an order manager 133. When the executable instructions are provided to processor 131, this causes processor 131 to perform operations discussed herein and below 133.
Each drive-through 140 can include a microphone, a speaker, and digital display or digital sign. Cameras 150 can be integrated into the drive-throughs 140 or can be separate standalone cameras 150 that are focused on the drive-through areas of the corresponding store.
During operation of system 100, images captured by camera 150 are made available through a common network accessible location or through direct streaming to order service 113. Order service 113 monitors the images for individuals, bicycles, cars, scooters, and other PEVs, when these objects are detected from the images. Computer vision or a MLM 114 can be trained to receive images and provide as output an identifier for an object, the object identifier associated with a car, a bicycle, a scooter, an individual walking, or PEVs. Order service 113 also assigns a drive-through lane identifier and store identifier to the corresponding image where an object was identified from the images. This can be based on a camera identifier for the corresponding camera 150, the camera identifier associated with a given store and a given drive-through lane of the store. Order service 113 uses the store identifier to identify the POS 130 of the corresponding store and sends a notification and/or a written descriptive object name and lane identifier with a time stamp to the corresponding order manager 133 in real time.
This allows order manager 133 to determine whether a customer is in a proper drive-through lane at the store and if not play an automated message through a speaker of the corresponding drive-through 140 to the customer indicating that the customer is in an incorrect drive-through lane along with instructions as to where the proper lane for placing an order is. Alternatively, order manager 133 raises an alert to staff, who operate a terminal associated with the POS 130, that instructs the staff to audibly notify the customer through a speaker of the drive-through 140 that the customer is in an incorrect drive-through lane along with audible instructions as to where the customer can find the proper drive-through lane to place an order.
In an embodiment, drive-through lane rules for the stores and their drive-throughs 140 are provided to order service 113 along with network access to the speakers associated with their drive-throughs 140. Here, order service 113 plays an audible message through the speakers informing the customers when they are not in the proper drive-through lanes along with instructions for ordering in the proper lane based on whether the customer is in a car or not in a car.
Assuming the customer is identified as being in the proper lane and the customer is not associated with a car, the image is passed to a second trained MLM 114 by order service 113 as input. The second MLM 114 is trained to provide written descriptive information for bikes, scooters, and PEVs along with written descriptive information of the customers. In an embodiment, the object type determined above as a car, bike, scooter, PEV, or walkup individual may be provided as input with the image to the second MLM 114. This allows for more focused training of the second MLM 114 to provide more fine-grain written descriptive output of the vehicle, if any, and the customer from the image by using an already determined object type, which was determined by the first MLM 114.
Order service 113 sends the lane identifier, the image, and the written description of the vehicle, if any, and the customer back to order manager 133. Order manager 133 links it to a current order being taking by staff at the store for the corresponding lane. Thus, the workflow associated with the order taken by the staff includes the image and written description. This allows the pay window and pickup window associated with the lane to identify the proper order to obtain the proper payment from the customer and to deliver the proper ordered items for the order to the customer.
In an embodiment, order service 113 indirectly interacts with the order managers 133 associated with the POSs 130 through the corresponding order systems 123 of the corresponding restaurant servers 120. Here, order service sends the object type identified, the store identifier for the store, the lane identifier for the drive-through lane, and the written description of the vehicle, if any, and the customer to order system 123 along with a time stamp. Order system 123 provided to the corresponding order manager 133 and integrates it into the order workflow for an order of a customer at a given store.
In an embodiment, a given store may include two drive-throughs 140 one for cars and one for non-cars but may only include a single drive-through pickup window. In these cases, order manager 133 instructs the customer to wait until called to come in, and pickup their order. The customer may be instructed to wait in a designated outside area where a speaker is available for staff to indicate when the non-car orders are ready for pickup. Alternatively, a separate pickup window that is not in the road around the store can be established by the store where the customer is instructed to come to pickup their order. The staff providing the order uses the linked written description of the customer to properly match the customer to their order.
In an embodiment, the written description produced by the second MLM 114 can include type and color of clothing, estimated height of the customer, any eyeglasses, etc. The written description can also estimate an age of the individual.
In an embodiment, order service 113 and MLM(s) 114 are provided through restaurant server 120. In an embodiment, order service 113 and MLM(s) 114 are provided onsite on a POS server 130 of a given store/restaurant.
System 100 permits stores or restaurants to establish at least two outdoor drive-throughs 140 for taking customer orders. Images captured by cameras 150 permit rapid identification as to whether for safety reasons a given customer is in a proper order lane and if not, the customer is notified to move to the proper lane. The images are also processed by a MLM 114 to obtain a written description of the customer's vehicle, if any, and a written description of the customer. The written description is linked to the order number of the customer by the corresponding order number, which permits staff at a pickup window and/or payment window to properly identify and provided the customer the proper ordered items.
System 100 can be integrated into existing car-based drive-throughs 140 by simply adding a camera 150 or by providing access to order service 113 to images of any existing cameras 150. The object identification and the written description of a customer's vehicle, if any, and a written description of the customer can be provided through an application programming interface (API) and integrated into existing order workflows associated with order manager 133 and/or order system 123 to receive the object identification, lane identifier, and written description from order service 113.
The embodiments of
In an embodiment, the device that executes the cloud-based drive-through order service is cloud 110 or server 110. In an embodiment, the device that executes the cloud-based drive-through order service is restaurant server 120. In an embodiment, the device that executes the cloud-based drive-through order service is a POS server 130 or a POS terminal of a given restaurant. In an embodiment, the cloud-based drive-through order service is order service 113 and/or MLMs 114.
At 210, the cloud-based drive-through order service obtains an image of a drive-through lane. The image can be obtained or received in a variety of manners.
In an embodiment, at 211, the cloud-based drive-through order service obtains the image in real time from a camera focused on an area associated with the drive-through lane. In an embodiment, at 212, the cloud-based drive-through order service obtains the image from a network-storage location. A camera focused on an area associated with the drive-through lane streams the image in real time to the network storage location.
At 220, the cloud-based drive-through order service determines an individual is present in the drive-through lane from the image. This can be done in addition to any car-based metal detection mechanism used with a conventional car drive-through lane using a camera focused on the drive-through area.
In an embodiment, at 221, the cloud-based drive-through order service associated a drive-through a drive-through lane identifier for the drive-through lane with the image. This is based on a camera identifier for a camera that captured the image.
At 230, the cloud-based drive-through order service identifies a the individual as a pedestrian present within the image. That is, the individual may be walking and have no vehicle. When a vehicle is present it can be identified as a car (gas or electric), a bike, a scooter, or any other PEV.
In an embodiment of 221 and 230, at 231, the cloud-based drive-through order service provides the image to a MLM 114 as input and receives as output from the MLM 114 the type of vehicle. It is noted that when a vehicle is not present the MLM may return a reserved type that indicates no vehicle was detected.
At 240, the cloud-based drive-through order service provides a lane identifier for the drive-through lane to an order manager 133 associated with the drive-through lane. In an embodiment of 231 and 240, at 241, the cloud-based drive-through order service plays an automated verbal instruction over a speaker of the drive-through lane to the individual that instructs the individual to move from the drive-through lane to a second drive-through lane based on the type and a rule assigned to the drive-through lane identifier maintained by the cloud-based drive-through order service.
In an embodiment, at 250, the cloud-based drive-through order service generates a first written description of a non-car vehicle, if any, and a second written description of the individual from the image or from a second image of the individual. The cloud-based drive-through order service provides the first written description when present and the second written description to the order manager 133 for an order placed by the individual through the drive-through lane or thru a different drive-through lane. That is, in some cases the individual in the first image may be in the incorrect lane such that the individual moves to a proper lane where a second image is taken.
In an embodiment of 250 and at 251, the cloud-based drive-through order service provides the image or the second image as input to a MLM 114 and receives the first written description when the non-car vehicle is present and the second written description as output from the MLM 114. In an embodiment of 251 and at 252, the cloud-based drive-through order service provides the type of non-car vehicle when present as additional input to the MLM 114.
In an embodiment, at 260, the cloud-based drive-through order service (210-240) is processed as a cloud-based service to the order manager 133. In an embodiment of 260 and at 261, the cloud-based drive-through order service integrates receiving of a type of vehicle and the lane identifier by the order manager 133 into a workflow associated with the order manager 133 through an API.
In an embodiment, the device that executes the order assistance manager is cloud 110 or server 110. In an embodiment, the device that executes the order assistance manager is restaurant server 120. In an embodiment, the device that executes the order assistance manager is a POS server 130 or a POS device/terminal onsite at a given store/restaurant.
In an embodiment, the order assistance manager is all or any combination of order service 113, MLM(s) 114, and/or method 200. The order assistance manager presents another and, in some ways, an enhanced processing perspective from that which was discussed above with respect to system 100 and method 200.
At 310, the order assistance manager identifies a non-car vehicle present in a non-car drive-through lane of a restaurant from an image taken of an area associated with the non-car drive-through lane. The order assistance manager initiates after an individual in a non-car vehicle is identified from the image in the non-car drive-through lane.
In an embodiment, at 311, the order assistance manager provides the image as input to a first MLM 114 and receives as output a type of non-car vehicle. The first MLM 114 trained to recognize from images non-car vehicles and provide their types as output.
At 320, the order assistance manager generates a first written description of the non-car vehicle and a second written description of an individual associated with the non-car vehicle from the image. The first written description may include the type of non-car vehicle and its color; the second written description may include a coarse-grain description of the individual such as height, color of clothing, any hat, and glasses, etc.
In an embodiment of 311 and 320, at 321, the order assistance manager provides the type of the non-car vehicle and the image as input to a second MLM 114 and receives as output the first written description and the second written description. In an embodiment, the second MLM 114 may be processed from a remote server as an artificial intelligence (AI) image-to-text service to obtain the first and second written descriptions.
In an embodiment, at 322, the order assistance manager provides the image as input to a MLM 114 and receives as output the first written description and the second written description. So, 2 MLMs 114 can be used in embodiments 311 and 321 whereas just 1 MLM 114 is used in the embodiment of 322.
At 330, the order assistance manager integrates the first written description and the second written description into order details for an order placed by the individual in the non-car drive-through lane. This integration can occur in a variety of manners.
In an embodiment, at 331, the order assistance manager sends a lane identifier for the non-car drive-through lane, the first written description, and the second written description to an order manager 133. The order manager 133 executes on a POS terminal 130 or a POS server 130 that takes the order of the individual.
In an embodiment, at 332, the order assistance manager sends a lane identifier for the non-car drive-through lane, the first written description, and the second written description to an order system 123 of a restaurant server 120 that interacts with an order manager 133. The order manager 133 executes on a POS terminal 130 or a POS server 130 that takes the order of the individual.
In an embodiment, at 33, the order assistance manager links the first written description and the second written description to an order number associated with the order details within a workflow of an order manager 133. The order manager 133 executes on a POS terminal 130 or a POS server 130 that takes the order of the individual.
It should be appreciated that where software is described in a particular form (such as a component or module) this is merely to aid understanding and is not intended to limit how software that implements those functions may be architected or structured. For example, modules are illustrated as separate modules, but may be implemented as homogenous code, as individual components, some, but not all of these modules may be combined, or the functions may be implemented in software structured in any other convenient manner.
Furthermore, although the software modules are illustrated as executing on one piece of hardware, the software may be distributed over multiple processors or in any other convenient manner.
The above description is illustrative, and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. The scope of embodiments should therefore be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
In the foregoing description of the embodiments, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting that the claimed embodiments have more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Description of the Embodiments, with each claim standing on its own as a separate exemplary embodiment.
