Patent Application
20240257094
In a typical vision-based checkout scenario, a customer places their items on a tray, images of the items are captured, the items are recognized using the captured images, and the customer uses a personal identification number (PIN) pad or card reader to complete the transaction. While vision-based checkout approaches seek to make the customer experience more frictionless, payment processing—which requires customer operation of the PIN pad and/or card reader at the checkout—remains a point of friction.
In various embodiments, methods and a system for frictionless vision-based checkouts are presented. A mobile application converts loyalty and/or payment details for a payment of a shopper into a code based on a store of a shopper. The application modifies the code with a prefix or postfix string of information. A transaction manager of a vision-based checkout system recognizes the modified code as a payment request of the shopper. The code can be displayed on the shopper's phone, and the phone can be placed on a tray alongside the items being purchased by the shopper such that the displayed code is visible to a camera or a barcode reader of the vision-based checkout system.
According to embodiments of the technology disclosed herein, an enhanced vision-based checkout system and methods are provided that recognize the phone as a non-salable personal item of the shopper, recognize a code displayed on the phone as being representative of a payment request, and provide the displayed code to the transaction manager. In some embodiments, after receiving confirmation from the vision-based checkout that all items are accounted for in the transaction, the transaction manager displays a countdown clock on a display of the self-checkout to the shopper. If the shopper takes no action, the code displayed on the phone is decoded to obtain payment card details and select a payment service associated with the payment card type. The transaction manager processes the payment through the payment service and the transaction completes in a frictionless manner (i.e. without the shopper being required to operate the card reader or the PIN pad to provide payment for the transaction). In an embodiment, the code when decoded by the transaction manager also identifies a store loyalty account for the shopper, which the transaction manager uses to record the appropriate loyalty credits for the transaction in the shopper's loyalty account.
While vision-based checkout scenarios seek to reduce friction in the customer checkout experience, payment processing remains a point of friction as it requires the shopper to operate the card reader of the terminal and/or the PIN pad to provide payment information to complete the transaction. The techniques presented herein and below provide a mobile application that allows a shopper to register various payment methods for use in stores to provide a frictionless payment experience for the shopper. The application generates a code that includes a prefix or postfix string of information that identifies the code as a payment code and that further identifies the type of payment being used. The code with the prefix or postfix information is displayed on a user-operated device, and responsive to the code being identified in an image or a scan of the display of the user-operated device, the transaction manager of the terminal decodes the code and selects the payment service used by the store for the payment type. The transaction manager uses the decoded payment details to process a payment for the transaction. As such, shoppers are provided a frictionless payment experience whereby they can place items on a tray of a vision-based checkout system alongside the payment code being displayed on the display of a mobile device to enable a payment without requiring the shopper to interact with a PIN pad or card reader.
Most shoppers have some form of digital payment available to them at all times (e.g., a credit card, a digital wallet, digital payment services, etc.). Some of these payment forms can be digitized into a barcode that can be shown via a phone of the shopper. The barcode is coarse enough to be recognized with an overhead vision checkout camera or scanned with a barcode scanner during checkout.
As used herein the terms, “shopper,” “consumer,” “customer,” and/or “user” may be used synonymously and interchangeably herein and may refer to an individual who is performing a checkout at a transaction terminal utilizing the frictionless payment techniques described in detail herein.
Moreover, various components are implemented as one or more software modules, which reside in non-transitory storage and/or hardware memory as executable instructions that when executed by one or more hardware processors perform the processing discussed herein and below.
System 100 includes a cloud 110 or a server 110 (hereinafter just “cloud 110”), user-operated devices 120, transaction terminals 130, and payment servers 140. Cloud 110 includes one or more processors 111 and a non-transitory computer-readable storage medium 112 (herein after just “medium”), which includes instructions for a wallet manager 113 and a registration manager 114. The instructions when provided to processor 111 cause processor 111 to perform operations discussed herein and below with respect to 113-114.
Each user-operated device 120 includes one or more processors 121, a display 122, and medium 123, which includes instructions for a wallet manager 124 and registration interface 125. The instructions when provided to processor 121 cause processor 121 to perform operations discussed herein and below with respect to 123-125.
Each transaction terminal 130 includes one or more processors 131, cameras 132, and medium 133, which includes instructions for a transaction manager 134 and an item recognizer 135. The instructions when provided to processor 131 cause processor 131 to perform operations discussed herein and below with respect to 134 and 135.
Each payment server 140 includes one or more processors 141 and medium 142, which includes instructions for a payment service 143. The instructions when provided to processor 141 cause processor 141 to perform operations discussed herein and below with respect to 143.
Initially, registration manager 114 interacts with a shopper through registration interface 125 for purposes of registering payment methods of the shopper. These payment methods can include credit cards, credit cards specific to particular retailers, known as store-specific cards, digital wallets, payment services, Apple Pay®, etc. Digital wallets (e.g., Apple Pay®) and payment services (e.g., PayPal®)
Separately, a retailer can register with registration manager 114 via a registration interface (not shown in
In some embodiments, registration manager 114 configures a wallet application (“app”) 124 on behalf of the shopper. The wallet app 124 may have previously been downloaded and installed on device 120 or can be configured, downloaded, and installed at the conclusion of registration, by registration manager 114. In the case where the wallet app 124 is pre-installed on device 120, the registration manager 114 may use an application programming interface (API) to configure wallet app 124 with the registered payment methods.
Following the appropriate registrations, system 100 operates as follows. A shopper places items for a vision-based checkout in a store at a terminal 130. Wallet app 124 can determine the store of the shopper based on a reported location of the device 120. Alternatively, a shopper operating a user interface of wallet app 124 can select the current store of the shopper. The registered payment methods available to the shopper are determined by the payment services 143 used by terminal 130 of the store. Thus, wallet app 124 presents available registered payment methods available from wallet app based on identification of the store, which can be automatically determined through location data or selected by the shopper through the user interface.
The shopper selects a payment method from a list of available payment methods through the user interface of wallet app 124. This causes wallet app 124 to generate a code the encodes the payment details of the selected payment method. The code is modified to include an encoded string of information that identifies the code as a payment token through a unique payment token identifier and that identifies the payment method being made. The string of information is prepended or appended to the code that encodes the payment details to create a modified code. Wallet app 124 displays the modified code on the display 122 of device 120. A sufficient amount of the available space on the display 122 is used to present the modified code to ensure that the modified code is captured by an overhead camera 132 of the terminal 130.
The modified code can also be set by wallet app 124 to be temporary such that it expires and is not displayable or usable after a preconfigured period of time. In such a case, the string of information appended or prepended to the code includes a date and time stamp for expiration. The string of information may also encode a number of permissible uses of the payment token, such that when transaction manager 134 encounters the payment token more than the threshold number of permissible uses, the payment token is ignored or is unusable.
Alternatively, the modified code is generated by wallet manager 113 when wallet app 124 reports the shopper's device location and/or the shopper's selected store and indicates that the shopper is requesting a payment token or a modified payment code (e.g., payment details with the prepended or appended string of information). Wallet manager 113 then generates the modified code and provides the modified code back to wallet app 124 for displaying on the display 122 of device 120.
The shopper places device 120 with the display facing up towards the overhead camera 132 of terminal 130 alongside the items that the shopper desires to purchase for the transaction. In example embodiments, transaction manager 134 of terminal 130 identifies the payment code and displays, on a terminal display, a message indicating that payment for the transaction will proceed in a predefined number of countdown seconds unless the shopper cancels the payment processing via a touch option presented on the terminal display to the shopper. The countdown timer may be configurable. Assuming the shopper does not cancel the payment, and at the end of the countdown timer, transaction manager 134 decodes the modified payment code to obtain the payment details and uses the payment details to interact with an appropriate payment service 143 to obtain payment for the transaction.
When the modified code is associated with a temporary payment token, transaction manager 134 compares the date and time stamp for expiration with a current date and time. Transaction manager 134 ignores the modified code if the current date and time are past the date and time stamp for expiration. Transaction manager 134 may present a message on the transaction display indicating that the payment token is expired and/or is unusable as payment for the transaction. In some embodiments, the payment code may encode a permissible number of uses within the string of information. For example, if the number of uses is set to 1, the payment token is usable just once. Transaction manager 134 can report the use of the token to a store server, such that each terminal 130 of a store can compare the payment token's number of uses against the number of permissible uses each time the modified code is encountered and ensure that only the authorized number of uses are allowed for transaction payments.
Item recognizer 135 is trained to identify user devices 120, such as phones, tablets, watches, etc. When item recognizer recognizes a user device 120, an image of the device is provided to transaction manager 134. Transaction manager 134 decodes the image provided by recognizer 135 and identifies the modified code as a payment code based on the prepended or appended string of information. The transaction manager 134 further identifies the type of payment and the payment details for the payment type from the payment code. This causes transaction manager 134 to enter a payment state for the transaction after a countdown timer is completed unless a cancel input is received from the shopper.
Item recognizer 135 is also trained to identify each item placed on a tray under the cameras 132. Each item identifier is provided to transaction manager 134 for processing with the transaction.
Additional embodiments and details are now continued with reference to
Diagram 100-1 depicts a user device 120 with a user interface for wallet app 124, operations performed when the shopper selects the generate quick response (QR) code option from the user interface, and a resulting payment token in the form of a QR code 126 depicted on the display 122 of device 120 after the operations are performed. The shopper can be presented with selectable payment method option(s) based on the location of device 120. Alternatively, the shopper may select a payment method within a user interface screen that precedes the user interface displaying the card details. The operations can include looking up a loyalty identifier for the shopper in a loyalty system of the store, obtaining a payment key for the card details, and generating a temporary payment barcode. The string of information that identifies the code as a payment token and that further identifies the payment type for the payment card details may be prepended or appended to the barcode and encoded as a QR code 126 displayed by wallet app 124 on display 122 of device 120. The QR code 126 includes the string of information and the card details and the QR code 126 can be generated by wallet app 124 or by wallet manager 113 on behalf of the wallet app 124. In some embodiments, a first string of information may be prepended to the code to identify it as a payment code and a second string of information may be appended to the code to identify the payment type, or vice versa.
In an embodiment, the string of information that is appended or prepended to the payment code can include a loyalty account number of the shopper. The shopper can provide the loyalty account during registration. The transaction manager 134 identifies the loyalty account and looks up the shopper's account during a frictionless payment and credits the transaction details to the shopper's loyalty account.
In an embodiment, the payment details are non-card based such as when the payment method selected by a shopper is a payment service 143 (e.g., Venmo®, etc.). In these cases, the payment details encoded in the payment code are details needed by the payment service 143, such as an email address, phone number, etc. This assumes the store accepts the payment service 143 for a given shopper's frictionless transaction.
In an embodiment, the payment code is provided via a near field communication (NFC) transmission from the user device 120 to the terminal 130. This can be done via a tap of device 120 on a designated location terminal 130 having an NFC transceiver.
In an embodiment, the payment code is scanned by the shopper after the shopper scans item codes. In this embodiment, the transaction is not a vision-based checkout transaction. The shopper, however, is still not required to touch a surface of the terminal 130 to perform the transaction since the shopper passes item codes for the items over or in front of the scanner to record the items being purchased and passes the payment code on display 122 of device 120 over or in front of the scanner to initiate payment on the terminal 130.
In an embodiment, wallet app 124 is subsumed into a specific retail store app as an enhancement to the retail store app. In an embodiment, the operations and functions of wallet app 124 are provided within an existing wallet app as an enhancement to the existing wallet app.
In an embodiment, the terminal 130 is a vision-based checkout system, a self-service terminal (SST), or a point-of-sale (POS) terminal. In the case of a POS terminal, the shopper can scan the payment code when the cashier indicates payment is needed by holding the display 122 out for the cashier to scan with a handheld scanner.
System 100 permits frictionless payments during checkouts. Transaction managers 134 are enhanced to identify a string of information prepended to or appended to a payment code. The payment code is identified by the transaction manager 134 as a payment token of a specific payment type. The payment token causes the transaction manager 134 to enter a payment state for the transaction. In an embodiment, the transaction manager provides a countdown before the state is moved to the payment state. The payment details are used by the transaction manager 134 to contact the appropriate payment service 143 and obtain payment for the transaction. This is a frictionless payment because the shopper does not have to operate the touch display of the terminal, a card reader of the terminal, or a PIN pad of the terminal to provide payment. The payment code can be provided during vision-based checkouts, self-checkouts, and/or cashier-assisted checkouts for frictionless payment.
The embodiments of
In an embodiment, the device that executes the frictionless payment service is cloud 110. In an embodiment, the device that executes the frictionless payment service is server 110. In an embodiment, the frictionless payment service is all or some combination of 113 and/or 114, as discussed above with system 100.
At 210, the frictionless payment service receives a request for a payment code. In an embodiment, the request is received from wallet app 124.
In an embodiment, at 211, the frictionless payment service receives a store identifier for a store associated with a terminal 130 at which a shopper is performing a vision-based checkout transaction.
In an embodiment, at 212, the frictionless payment service receives a current location of a mobile device 120 of the shopper. The frictionless payment service obtains a store identifier for a store associated with the terminal 130 based on the current location of device 120 and a known location associated with the store or the terminal 130.
At 220, the frictionless payment service obtains payment details for a payment method. The payment details may vary based on the type of payment method. For example, the payment details for a credit card may include account number, billing zip code, security code, etc. The payment details for a payment service (e.g., payment service 143) may include an email or user name, a phone number, etc.
In an embodiment, at 221, the frictionless payment service selects the payment method based on a store associated with the terminal 130. That is, a store identifier is linked to the payment service 143 used by the store. A given payment service 143 accepts only certain types of payment methods. Registered payment methods associated with the shopper are used to select the payment method that comports with the store's payment service 143.
In an embodiment, at 222, the frictionless payment service identifies the payment method based on a selection made by the shopper through an interface that identifies the store associated with the terminal 130. In an embodiment, the interface is the user interface to wallet application 124.
At 230, the frictionless payment service generates a payment token identifier for the payment code and identifies a payment type for the payment method. The identifier is recognizable by a transaction manager 134 of the terminal as identifying a payment token.
In an embodiment, at 231, the frictionless payment service assembles the payment token identifier and the payment type as a string of information. This string of information is prefixed or postfixed onto the payment details by the frictionless payment service. In an embodiment, the payment token identifier may be represented as a first information string prefixed to the payment details, and the payment type may be represented as a second information string postfixed to the payment details, or vice versa.
In an embodiment, at 232, the frictionless payment service encodes, in the payment code, the payment details with a loyalty identifier for a loyalty account of the shopper with a store associated with the terminal 130. In an embodiment, at 233, the frictionless payment service encodes, in the payment code, an expiration date and time of day. In an embodiment, at 234, the frictionless payment service encodes, in the payment code, a number representing a total number of times that the payment code is usable by the shopper.
At 240, the frictionless payment service encodes the payment token identifier, the payment type, and the payment details in a payment code. In an embodiment, at 241, the frictionless payment service encodes the payment code as a barcode or a QR code 126.
At 250, the frictionless payment service provides the payment code to the mobile device 120 of the shopper. The payment code is displayed on a display 122 of device 120 and presented at the terminal 130 for a frictionless payment during a transaction of the shopper. The transaction can be a vision-based checkout transaction, a self-checkout transaction, and/or a cashier-assisted checkout transaction. In an embodiment, at 251, the frictionless payment service instructs a wallet application 124 of the mobile device 120 to display the payment code on a display 122 of the mobile device 120.
The frictionless payment app interacts with the method 200. In an embodiment, the device that executes the frictionless payment app is user-operated device 120. In an embodiment, device 120 is a phone, a tablet, or a watch. In an embodiment, the frictionless payment app is wallet app 124.
At 310, the frictionless payment app receives a request for a payment code. At 320, the frictionless payment app identifies a payment method for the payment code.
In an embodiment, at 321, the frictionless payment app selects the payment method from a plurality of payment methods associated with the shopper based on a store associated with the terminal 130. In an embodiment, at 322, the frictionless payment app presents available payment methods to the shopper through a user interface of the mobile device 120 based on a store associated with the terminal 130. The frictionless payment app receives a selection made by the user from the available payment methods presented through the interface.
At 330, the frictionless payment app obtains payment details associated with the payment method. For example, the payment details may include a user account number, billing zip code, security code, email address, phone number, etc.
At 340, the frictionless payment app appends a string of information to the payment details. The string of information includes a payment token identifier and a payment type associated with the payment method. The payment token identifier allows a transaction manager 134 to identify the code as being representative of a payment request, with a payment type expected thereafter, and then followed by the payment details.
In an embodiment, at 341, the frictionless payment app adds a loyalty account associated with the shopper to the string of information. In an embodiment, at 342, the frictionless payment app adds an expiration date and time for the payment code and a total number of uses for the payment code to the string of information.
At 350, the frictionless payment app encodes the string of information and the payment details as the payment code. In an embodiment, at 351, the frictionless payment app provides the payment code as a QR code 126.
At 360, the frictionless payment app renders the payment code on a display 122 of the mobile device 120 for a shopper to present to the terminal 130 for frictionless payment during a transaction at the terminal 130. That is, an image of the payment code or a scan of the payment code is sufficient for a transaction manager 134 of terminal 130 to enter a countdown for transitioning to a payment state and to process the payment for the shopper using the payment details when the countdown expires.
In an embodiment, at 370, the frictionless payment app is processed as a wallet application 124 on the mobile device 120. The wallet application 124 provides a user interface for the shopper to register payment methods as discussed above with system 100.
It should be appreciated that where software is described in a particular form (such as a component or module) this is merely to aid understanding and is not intended to limit how software that implements those functions may be architected or structured. For example, modules are illustrated as separate modules, but may be implemented as homogenous code, as individual components, some, but not all of these modules may be combined, or the functions may be implemented in software structured in any other convenient manner.
Furthermore, although the software modules are illustrated as executing on one piece of hardware, the software may be distributed over multiple processors or in any other convenient manner.
The above description is illustrative, and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. The scope of embodiments should therefore be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
In the foregoing description of the embodiments, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting that the claimed embodiments have more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Description of the Embodiments, with each claim standing on its own as a separate exemplary embodiment.
