Apparatus and method for payment of a texting session order from an electronic wallet

TECHNICAL FIELD

The present invention relates to customer communication technology and, more particularly, to artificial intelligence assistants.

BACKGROUND ART

Retail establishments have numerous technologies at their disposal for improving the customer experience. U.S. Pat. No. 8,775,260 discloses user interfaces fixed within a store where a user can inquire about the location of a product and receive a response indicating the product's location. U.S. Pat. No. 9,147,212 discloses a mobile phone system in which a user makes a voice call to an automated voice response system that can provide a product location in response to a voice inquiry. U.S. Pat. No. 9,754,176 teaches a method of extracting data from a document such as a sales receipt. Photographs of receipts taken by a smartphone may be used to support customer loyalty programs. Timetrade advertises an ability to schedule appointments through text messaging.

While a retail establishment can expend its resources to build out any one or more of the available services, it involves a tremendous commitment. Reaching customers through various combinations of apps, voice calls, kiosks and texting services may serve to be confusing and counterproductive.

SUMMARY OF THE EMBODIMENTS

In accordance with one embodiment, an application server acts as an intelligent interface that includes an IP network interface associated with at least one phone number. Requests received, in which the phone number is a destination phone number, will also include a source phone number and message content. A reply generator in the intelligent interface generates a reply wherein the source phone number is used as the destination and the destination phone number is the source. The ability to respond to the message content is made possible with a first vectorizer that generates a serialized intent file and stores the file in memory. An intent classifier determines, based on the message content, one intent from the serialized intent file. A response generator is responsive to the one intent for producing a response. The response is provided in the reply.

According to one embodiment, the IP network interface is associated with a landline phone number. Embodiments may further include a gateway in communication with a cellular network and the IP network. The gateway is configured to convert cellular network communications to IP network communications and IP network communications to cellular network communications.

Embodiments may further include a second vectorizer in communication with a collection of intent object attributes. The second vectorizer is configured to create a serialized intent object file. The response generator has access to the serialized intent object file to assist with generating responses.

In a particular embodiment, the intent classifier is responsive to message content including a purchase receipt image to determine an intent corresponding to a loyalty program. In an additional particular embodiment, one of the intents in the serialized intent file is a product locator program.

According to a method embodiment, a serialized intent file having a plurality of intents is provided. A request including a source phone number, a destination phone number and message content is received from over an IP network. Based on the message content, an intent from the plurality of intents is determined. In accordance with a program corresponding to the intent, response message content gets generated. A reply including the source phone number as a destination, the destination phone number as a source and the response message content is transmitted over the IP network.

Intents may correspond to programs leading to any of a variety of operations. The program may be an app. The program may lead to retrieving a resource. The resource may be any of a variety of types. The resource may be a link to a web application or a link to a web page. According to a further alternative, the resource may be a uniform resource identifier for launching an app or utility on a mobile communication device. A further program type is a predefined response script.

Additional steps may typically be performed by a gateway. A text message received over a cellular network is converted to the request sent over the IP network. In the other direction, the reply is received over the IP network and the response message content is forwarded in a text message to the destination.

Customer utilization of intelligent text messaging sessions can be significantly increased in accordance with the addition of a voice call conversion system to the present invention. In particular, the system coordinates with a telephony server to provide a pathway to encourage interaction through intelligent texting sessions. A telephony server may include a call control module in communication with each of a calling line ID interface, a text-to-speech module, a speech-to-text module and a DTMF decoder module. The call control module is configured to generate and send over the IP network a request to the application server, wherein the request includes a source phone number determined by the calling line ID interface or otherwise entered by the caller, message content determined by the speech-to-text module or the DTMF decoder and a client identifier. The application server is equipped with an interactive voice response app configured to control interactions by the telephony server with a caller over the telephone network and to provide the caller with an option to request a text message session. Upon receiving a request for a text message session, the application server commences a session according to a program corresponding to the request by generating a reply that includes the source phone number as a destination, a destination phone number corresponding to the client identifier as a source and message content determined by the program.

Insofar as the application server is concerned, voice call conversion begins with receiving a request over an IP network from a telephony server to initiate an interactive voice response session with a caller who placed a voice call to a destination phone number. The application server instructs the telephony server to produce a voice inquiry over the telephone network to the caller, wherein the voice inquiry offers the caller an option to request a texting session. When a request for a texting session is received from the telephony server, the application server generates message content in accordance with a program corresponding to the requested texting session. A reply is transmitted by the application server over the IP network. The reply contains text message components including the phone number as a destination, the destination phone number as a source and the message content.

Reviewing the operation of the overall system for voice call conversion, the telephony server first intercepts a voice call to a destination phone number. A request is sent over the IP network from the telephony server to the application server to initiate an interactive voice response session with a caller who placed the voice call over the telephone network. In response to instructions from the application server, a voice inquiry is produced by the telephony server and transmitted over the telephone network to the caller. The voice inquiry offers the caller an option to request a texting session. When a request for a texting session is received by the telephony server, the telephony server sends a corresponding request over the IP network to the application server. The application server generates message content for delivery by text message, in accordance with a program corresponding to the requested texting session. The application server transmits a reply to the request for a texting session in which the reply includes the phone number associated with the caller as a destination, the destination phone number as a source and the message content. A gateway on the IP network receives the reply and forwards the message content in a text message to the destination.

According to one embodiment, an intelligent interface responds to an order received in a text messaging session by initiating payment from an electronic wallet. An IP network interface is configured to receive requests including a source phone number, a destination phone number, and message content. A response generator is configured to generate response message content responsive to the message content in one of the requests. A reply generator is configured to transmit a reply over the IP network including the source phone number as the destination, the destination phone number as the source, and the response message content. An ordering program runs on behalf of the response generator, the ordering program is configured to receive an order destined to a client establishment associated with the destination phone number. The ordering program is further configured to provide, as response message content, payment options from an electronic wallet associated with the source phone number. Selection of at least one of the payment options causing processing of the payment for the order from the electronic wallet. The ordering program is configured to then forward the order to the client establishment associated with the destination phone number.

In some embodiments, the intelligent interface also includes an intent classifier configured to receive the message content from the IP interface, and determine, based on the message content, intent from a plurality of intents in a serialized intent file. In some embodiments, the ordering program is one of the plurality of intents.

In some embodiments, the requests may include any of a SMS message, MMS message, Rich Communication Services (RCS) message, and iMessage. In further embodiments, the response message content may include a web payment URI that displays the payment options via a web browser on a mobile device associated with the source phone number. In some embodiments, one or more of an offer, a promotion, a loyalty program, or a rewards program is stored in the electronic wallet. In further embodiments, the ordering program is configured to store a receipt or other record of the order in the electronic wallet. In particular embodiments, the destination phone number corresponds to a landline.

In some embodiments, the intelligent interface further includes a database configured to store a plurality of electronic wallets, each associated with a phone number of a mobile device. In some embodiments, the intelligent interface also includes a gateway in communication with a cellular network and the IP network, the gateway is configured to convert cellular network communications to IP network communications and IP network communications to cellular network communications.

According to a method embodiment, requests including a source phone number, a destination phone number, and message content are received over an IP network. An order destined to a client establishment associated with the destination phone number is received by the requests. Response message content responsive to the message content in one of the requests is generated, wherein the response message content includes payment options from an electronic wallet associated with the source phone number. A reply is transmitted over the IP network including the source phone number as the destination, the destination phone number as the source, and the response message content. In response to selection of at least one of the payment options, causing processing of the payment for the order from the electronic wallet. The order is then forwarded to the client establishment associated with the destination phone number.

In some embodiments, based on the message content, intent from a plurality of intents in a serialized intent file is determined, and an ordering program is run, when the intent corresponds to an ordering program. In further embodiments, a plurality of electronic wallets are stored, each associated with a phone number of a mobile device. In some embodiments, a plurality of phone numbers are stored, each associated with at least one of a plurality of client establishments served by the intelligent interface.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing features of embodiments will be more readily understood by reference to the following detailed description, taken with reference to the accompanying drawings, in which:

FIG. 1 is a block diagram illustrating a system including an intelligent interface and voice call conversion system of the present invention.

FIG. 2 is a flowchart of a program corresponding to a shopping list intent for use in the intelligent interface of FIG. 1.

FIG. 3 is a flowchart of a program corresponding to a consumer survey intent for use in the intelligent interface of FIG. 1.

FIG. 4 is a flowchart of a program corresponding to a loyalty program intent for use in the intelligent interface of FIG. 1.

FIG. 5 is a flowchart of a program corresponding to an order taking intent for use in the intelligent interface of FIG. 1.

FIG. 6 is a flowchart of a targeted offer program for use in the intelligent interface of FIG. 1.

FIG. 7 is a flowchart of an interactive voice response method for use in FIG. 1.

FIG. 8 is a flowchart of a voice call conversion method performed at an application server.

FIG. 9 is a flowchart of a voice call conversion method performed by the system illustrated in FIG. 1.

FIG. 10 is a flowchart of a payment method for use in the intelligent interface of FIG. 1, in accordance with an embodiment of the present invention.

FIG. 11 is a display presented by the payment method of FIG. 10, in accordance with an embodiment of the present invention.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Definitions. As used in this description and the accompanying claims, the following terms shall have the meanings indicated, unless the context otherwise requires:

A “text message” as used herein is an electronic communication sent and received by a network connected device, such as a mobile phone or tablet, that includes SMS messages and MMS messages. Versions of these messages include Android's Rich Communication Services (RCS) and Apple's iMessage.

The term “memory” as used herein includes all media in a system that can be used to store data including nonvolatile storage, volatile storage, ROM, disk, solid state memory, RAM, internal storage and external storage.

The term “server” as used herein is open-ended and includes a single server or a plurality of servers operating as a server system.

FIG. 1 illustrates an application server 100, referred to herein as an intelligent interface, configured to provide any of a variety of services for any of a number of clients to the users of phones, particularly, phones with text messaging capability. An intelligent interface in accordance with the invention may be built on one or more servers. Clients may include retail stores, restaurants, insurance brokers, real estate brokers, or any service providers who wish to improve communications with customers or users of their service. Each client has one or more phone numbers for receiving text messages to be handled by the intelligent interface 100. In accordance with a particularly advantageous embodiment of the present invention, the phone number may be a landline phone number for the client establishment. By using the landline phone number, the establishment benefits from using a number that has been long associated with the establishment and may be located in a number of phone directories and on websites. The intelligent interface 100 thus leverages the value and extends the technical reach of a communication channel already associated with the establishment.

Each client has a portal 102a,b,c through which it can communicate with the intelligent interface 100 over the IP network. Through the portal, the client can set up selected services to be made available to users of its phone numbers. Services may include any of a predefined scripted response, resource responses and interactive app responses. The client can make a customized service available by providing the intelligent interface 100 with data corresponding to the establishment. Some exemplary data frames that an establishment might provide include a data frame listing all of the products and their location on store shelves, a data frame listing all cars for sale on a lot, or a data frame listing all menu items and their prices.

The various available services are stored in an intents catalog 122 in the memory 120 associated with the intelligent interface 100. When receiving a request originating from a text message device, the intelligent interface identifies the service from among those available to the called client which most closely matches the intentions of the requester. Thus, the services are called intents. For example, intents may be directed to helping a user with locating a product, navigating a loyalty program, making a reservation, seeking a job, learning about events, organizing a shopping list, navigating the available intents, answering a common question, providing feedback to an establishment, ordering or using gift cards, receiving a targeted discount offer and getting help with the intelligent interface. Other intents may be added to the intents catalog as they are developed. The intelligent interface 100 advantageously provides a wide variety of services that its clients can make available to their customers or users.

The intents catalog 122 includes the intents and a compilation of language and visual data indicative of a request for that intent. The compilation can be manually generated. The compilation can be supplemented by information provided by the client.

Some intents may make use of additional information in a request, for the purpose of identifying objects that particularize the request. For example, in selecting an item from a menu the request may use language recognizable but not identical to the language on the menu. An objects catalog 124 includes the objects relevant to an intent and a compilation of language and visual data indicative of each object. Objects data frames may be populated with data provided by a client. A client may provide databases or files related to the services being provided to that client on the intelligent interface. For example, a listing of all products, their classification and their location when the product locator service is being made available should be provided by the client. Similarly, a restaurant client would provide a listing of all menu items to facilitate a meal ordering service. Many other examples are possible such as a listing of properties available for sale on behalf of a real estate broker establishment or a listing of vehicles for sale at an auto dealership.

The object data frames may be stored in a response configuration database or interchange data frame format. Thus, the object data frame may reside in a relational database dedicated to response configuration relationships. The object data frames are more broadly defined to include generic (interchange format) data frame files. The intelligent interface provider will need to administer the system. Client accounts are maintained requiring storage of account and user information, identity of selected services and usage data. Use of the various services may be monitored permitting the administrator to offer clients useful reports regarding their customer and user habits. The administration data is stored in the Account, Store and User Data database 126.

User accounts are maintained that store account information, such as one or more telephone numbers associated with each user, in the Account, Store and User Data database 126. Electronic wallets 129 are also maintained that provide access to one or more payment methods configured by the users, such as credit cards, Paypal, Bitcoin, etc. The electronic wallets 129 may also provide access to receipts or other records of users' past orders, redeemable offers and promotions (e.g., coupons) associated with client establishments, and loyalty and rewards programs in which the user is a member. The electronic wallet of a user is associated with one or more telephone numbers stored in the user's account. The one or more telephone numbers are used to send and receive text messages.

User sessions are tracked in the intelligent interface 100. A log of the requests and replies is maintained at least while the session remains active. App Data 128 corresponds to the app data and logs stored in memory 120. This includes working memory for various processes being performed to support an app or script as it is running. The app data 128 includes logs of request and reply sessions with users. Sessions remain active until they are completed or timed out. Inactive session data may be stored in an archive. Memory 120 also maintains data related to each of the clients including the services that have been selected and logs of communications with each client.

Each intent in the intents catalog 122 is associated with a program for responding to requests. The program is stored in memory 120 and may be considered as part of the app data 128. The intelligent interface 100 can advantageously offer different types of interactive services for association with any of the intents. One type of service is a predefined response script. The predefined response script merely produces a response to the request. For example, a “goodbye” intent may be programmed with a script that provides a suitable signoff response. A general information request intent may be programmed to provide an informational response with details as to address, phone, store hours, website and the like.

Another service that can be made available through the intelligent interface 100 is an application or “app”. An app is a program that interacts with the user through a series of queries and responses to perform a group of coordinated functions, tasks, or activities for the benefit of the user. The app may be built into the intelligent interface 100 or it may be created or customized by a client through its client portal. The apps may be stored among the app data frames 128 for access in response to customer or user requests.

When an unspecific greeting such as “hello” is the essence of the request, the “request opening app” intent may be activated to provide the user with a greeting and an explanatory screen to help the user select a particularly relevant and useful intent. A “help” intent may be programmed with an app that provides an options menu in response. The user responds with a selection to launch the indicated intent from the options menu.

Other intents may be associated with programs that retrieve a resource. The available resources offer different types of services. One type launches an app or a utility on the customer or user's text messaging device 140. A typical text messaging device 140 is a mobile smartphone. To launch an app or utility on a smartphone, the intelligent interface replies by sending along a uniform resource identifier (“URI”) internal to the phone for the specified app. The URI may be particular to the intent for the particular client. Thus, for example, the link may not only open an app on the text messaging device 140, but it may more specifically open a response page to a request embedded in the URI. The app data database 130 contains a message to go with the uniform resource identifier, so that the customer or user understands to click on the link upon receiving it in the text message.

Another type of service may be provided through a web application. The resource for the application is a link to the web application. The link is a URL that is fed to the browser on the smartphone when the user activates the link. Typically, links are activated by touching them with a finger or stylus. The user can then interact with the web application to obtain requested information or instructions or perform functions, tasks or activities associated with the application for the benefit of the user.

Another service may simply be a web page. The resource for the web page is a link to the web page. The link is a URL that is fed to the browser on the smartphone when the user activates the link. The web page may provide information helpful to the requesting user. For example, the page may provide a product description or a store map. With the intelligent interface 100, a client can set up desired services in association with specified intents to satisfy the needs of its customers and users.

Customers and users are advantageously able to access the services of a particular client through text messages to the intelligent interface 100. A gateway 104 is provided with the phone numbers designated by the clients for text message access by its customers or users. The gateway 104 intercepts text messages sent over a cellular network to any of the designated phone numbers. The designated phone number may be a cellphone number or a landline phone number, so long as it registered with the gateway in connection with the intelligent interface. When the gateway 104 receives a text message, it embeds the message in an HTTP request addressed to the intelligent interface 100. The HTTP request is sent out over the Internet Protocol network 106. There are a number of gateway providers who may provide the gateway function required by embodiments of the present invention. For example, Twilio, Inc. of San Francisco, Calif. offers such a gateway service.

The intelligent interface 100 receives the HTTP requests through its IP network interface 110. The request includes as a destination phone number, the phone number designated by one of the clients. The request further includes a source phone number and message content. The source phone number identifies the text messaging device user.

In order to be responsive to natural language inputs, the ability to interpret the request is critical. Successful operation of the intelligent interface 100 depends on converting the intents catalog and the objects data frames into serialized files. The conversion is performed by intent vectorizer 150 and object vectorizer 152. Vectorizers are known in the art. Any of the vectorizers used in the present invention may be a single vectorizer or an ensemble of vectorizers, in either case they are referred to herein as a vectorizer. A vectorizer associates each intent or object with a series of numbers each number giving a strength of relationship with one of a plurality of different words, phrases or images. A Bag of Words model is one approach to generating the relationships for vectorization. The model is applicable to images as well as words. By providing the vectorizer with numerous sample word or image inquiries representative of a particular intent, a vector associated with an intent can be generated for use in analyzing and classifying requests from users. In similar fashion, the object vectorizer generates a vector for an object. An example of a vectorizer is the TfidfVectorizer available online from scikit-learn.org. The TfidfVectorizer is equivalent to scikit's CountVectorizer followed by TfidfTransformer.

Building a serialized intent file for an intent classifier shall now be explained according to one particular embodiment. Intent data can be prepared in a JSON format where key is the intent and value is the list of keywords or phrases related to the intent. The JSON formatted data is loaded into a pandas.dataframe. The data frames are stored in the intents catalog 122. Training documents with sample text messages which can include any combination of textual and/or image messages are converted to vectors by a vectorizer. In advance of the vectorizer, a stemmer may be used to simplify the contents of a textual message. For example, an nltk.stemmer may be used. Further simplification can include removing or filtering out stop words. According to the TfidfVectorizer, the vectorizer may use an in-memory vocabulary (a Python dict) to map the most frequent words to features indices and hence compute a word occurrence frequency (sparse) matrix. The word frequencies are then reweighted using the Inverse Document Frequency (IDF) vector collected feature-wise over the corpus of training documents. Classification is based on the k-NN algorithm for the model developed by the training documents.

Upon vectorizing the intents and the objects, they are saved in memory in a serialized intent file 154 and serialized object file 156, respectively. For Python users, such files may be known as pickle files. The IP network interface 110 receives the HTTP request and it is passed to the response generator 116 which examines the source telephone number provided with the HTTP request. If an active session is in progress with that source for the client designated by the destination phone number, the response generator 116 returns to the program for the intent that is in progress and generates a next response in view of the program and the message received in the HTTP request. New requests, however, are first analyzed by an intent classifier 112.

The intent classifier 112 compares the request with the serialized intent file to determine an intent most closely related to the request. According to some embodiments, the model characterized by the serialized intent file uses a two dimensional plane. One axis is the intents and the other axis is the vectorized representations of the document. Each trained document is a point on the plane. The classifier may rely upon any of a number of known approaches including but not limited to generalized linear models, support vector machines, nearest neighbors, decision trees and neural networks. In a preferred embodiment, the intent classifier 112 vectorizes the request in a fashion similar to that used on the training documents and runs a nearest neighbor algorithm comparison between the request vector and those in the serialized intent file. The nearest neighbor vector identifies the intent that will be used for responding to the request. In preferred embodiments, the k-NN algorithm uses the Minkowski distance to determine the distance from the vector being classified and the other vectors on the plane. The closest vector is picked and its intent is selected.

Some intents have programs which are responsive to objects in the request. These programs will engage an object classifier 114 to identify an object in the request. The object classifier 114 can work in the same manner as the intent classifier. The classifier may rely upon any of a number of known approaches including but not limited to generalized linear models, support vector machines, nearest neighbors, decision trees and neural networks. The object classifier 114 can vectorize object attributes in the request. Vectorization may include parsing into parts of speech, extracting nouns or verbs and matching to stored object data. In a preferred embodiment, the object classifier determines the nearest match, by running a nearest neighbor algorithm comparison between the vectorized object attributes in the request and the vectors in the serialized object file. The nearest neighbor identifies the object that will be used in the program of the intent.

The response generator 116 determines the response to be sent back in a reply. The intent determines the applicable program to be followed by the response generator for determining the response. The response may be a predefined response from a script or may be a resource leading to any of a plurality of possible type services. For scripts, the response generator follows commands and provides responses according to the script stored with the app data 128 in memory 120. For in-progress requests received by the IP Network interface 110, the response generator reviews the state of the script in its log among the app data 128 and generates a next response in accordance with the script. For new requests, the indicated script is accessed and a response is generated as specified in the script.

With respect to resource responses, the program associated with the intent provides a suitable response message, which will include a link or a uniform resource identifier. The link may direct the user to a web page or a web application. The uniform resource identifier launches an app or utility program on the user's mobile communication device, typically a smartphone.

The generated response is ready for packaging into an IP network communication. Reply generator 118 completes the communication. The source phone number from the original HTTP request becomes the destination for the reply. The destination phone number from the original HTTP request becomes the source in the reply. The response and the phone numbers will constitute the text message sent to the user's text messaging device. The reply generator, thus advantageously, extends the technical reach of the intelligent interface 100 to text-messaging devices 140. In the IP network interface, the text message is appended to an IP packet addressed to the IP address of gateway 104. The IP packet is sent out by the IP network interface 110 over the IP network 106 where it is routed to the gateway 104. Gateway 104 extracts the text message and sends it over the cellular network to the requesting user.

The intelligent interface 100 advantageously makes numerous intents available to its clients. Some of these services may be existing services available online, which are now being made available through the unique text messaging interface incorporated into the intelligent interface 100. The inventors have also developed new services for access through the intelligent interface 100.

As shown in FIG. 2, a shopping list app may be associated with an intent. When a request supplies a list of items purchasable in the client establishment, the intent classifier 112 may determine that the shopping list intent is being requested 200. The shopping list intent is associated with a program that calls upon the object classifier 114. The object classifier 114 associates each item in the list with the closest item in the client establishment's object data 202. For each object classified in the list, the program identifies the product location in the object data for the client. The shopping list can then be sorted according to product location and returned to the user in one or more responsive text messages thereby providing the user with an ordered shopping list 204. For added convenience, the program may include each product's location when responding with the sorted list. A shopper can thus conveniently text his or her shopping list to the establishment's phone number and receive back a sorted list conveniently arranged by location.

The intelligent interface 100 further provides a convenient means for collecting feedback from customers. Referring now to FIG. 3, a customer at any time during or after shopping or dining at an establishment can seek to provide unstructured feedback or fill out a survey to report on his or her experience. The survey is readily accessed through the customer's handy text messaging device 140. Upon recognizing the user's intent is to provide feedback or respond to a survey, the feedback or survey app for the particular client is accessed 250. For unstructured feedback, the program analyzes the customer's message and extracts the relevant topics covered for later analysis and presentation to the merchant. Having extracted the topics covered in the feedback, the intelligent interface can generate a specific response to the customer acknowledging their specific feedback topics. For surveys, a question is sent in a reply text to the user 252. An HTTP request text from the user is received and the response given in the text is logged 254. The app process repeats sending survey questions and receiving responses from the user until the user stops responding or the final survey question is sent out 256.

Loyalty programs can be enabled or particularly enhanced by the text messaging intelligent interface 100. Referring now to FIG. 4, a loyalty program app shall now be described. The intent classifier 112 determines that a user request should activate the loyalty intent 300. A particularly useful way to activate the loyalty intent is for the user to snap a photo of his or her customer receipt and forward the photo by a text to the establishment's phone number used by the intelligent interface 100. In a preferred embodiment, that phone number may be a landline phone number associated with the establishment.

In order to facilitate photo activation of a corresponding app, the intent classifier should be configured to include image recognition. For example, the intent classifier could begin its analysis of a request by determining if the request contains visual data. If yes, an image recognition technique can be activated to classify the image or video. For purposes of the loyalty app of FIG. 4, a determination is made as to whether the visual data is a receipt. One available technique for evaluating visual data is to vectorize the image to translate the image format into one that can be evaluated algorithmically for a match. Scikit learn's kNN algorithm is one of the readily available programs for image recognition.

When the intent classifier 112 recognizes a receipt, the loyalty intent has been determined 300. The loyalty app then calls upon optical character recognition (“OCR”) 160. Transaction details are extracted from the receipt photo using the optical character recognition 302. A transaction validation process is initiated 304. Transaction validation can be performed in the intelligent interface or in communication with a third party application through a third party application interface 170. The application may be found in, for example, a loyalty program provider or a system of the client. Communications are typically exchanged through the IP Network Interface 110 over the IP network. The loyalty program app initiates an inquiry to the loyalty database to determine if the user's phone number identifies a valid loyalty program account 306. In some embodiments, the loyalty program account 306 may be accessed from an electronic wallet associated with the user's phone number. If the account is found 307, the transaction details are added to the account history 310. If the account is not found 309, the phone number is added as a new account 312. Alternatively, validation can be conducted in the intelligent interface, by for example, performing format/layout matching of the merchant's proof of purchase receipt. Indeed, the loyalty program itself can be hosted and stored at the intelligent interface without reliance upon a third party loyalty program provider. The transaction details are saved to a history for the new account. A response is sent back to the user to confirm that these actions were taken 314.

Referring now to FIG. 5, a flow for two related intents is shown. The intent classifier 112 receives an incoming shopper request 400. The request may include a destination phone number of the client establishment. When the intent classifier identifies 410 the request as a simple ordering request, the response generator will produce a corresponding inquiry for use as the response. The simple ordering intent calls for a response of “what would you like to order?” 412. Upon receipt of the user's order request 414, a confirmation response is transmitted to the user 470 and the order as texted by the user is sent through a third party application interface 170 to the client establishment where the order can be printed 480. Thus, a user is able to transmit an order via text message to a store, and in some cases, that text message can be addressed to the store's landline phone number.

If the intent classifier 112 identifies the request as an intelligent ordering intent, the object classifier 114 is prompted by the ordering app to determine if the request contains the needed information specifying the item, quantity and any modifications to the item being ordered 420. If any of the information is missing, the app causes a message to be generated requesting the needed information. (not shown) If the app cannot discern an order of an item from the request, it can send a request indicating its lack of comprehension. For example, a response reading “I didn't quite get that, continue order?” response can be sent to the user 424. The user can choose to give up and discontinue the order 426. Alternatively, the user can try sending an order with a next text message. The app will then try to read the new order text message to determine if it can identify an item to be ordered from the order objects 420.

When the information for an item has been obtained, the ordering app sends a request summarizing the order 430 and asking if the user wishes to order any other items 432. If yes, the process repeats itself to complete the ordering information for this next item. If the user is done ordering items, the user may be asked “Do you want to pay now?” 436. If not, a confirmation response 470 can be generated. The confirmation response preferably includes a listing of the items ordered. If the user is going to pay, the ordering app generates, based on the items ordered, and delivers a payment request URI 450. The request is sent in a reply to the user, arriving at the user's text messaging device via text.

In an embodiment, the ordering app generates and delivers the payment request URI 450 according to the method in FIG. 10. In this method, the ordering app initiates an inquiry to the account database to determine if the phone number used in the text messaging is associated with an electronic wallet 1005. Given that the identity of the user phone has been verified by having engaged in a texting session with the intelligent interface, no further verification is required to provide access to the electronic wallet. If an electronic wallet is not found 1007, a message is sent in the reply to the user 1009, which may request the user to configure an electronic wallet or provide an alternative method for the user to pay. If an electronic wallet is found 1010, the ordering app generates the payment request URI to include payment options from the electronic wallet 1015. The URI is sent in the reply to the user 1020, and the payment options are presented to the user 1025 via a web browser on the user's text messaging device. FIG. 11 shows an example presentation of the payment options 1110 by the payment request URI. Selection of at least one of the displayed payment options causes processing of the payment for the order from the electronic wallet 1030. The ordering app may also apply any offer, promotion, loyalty program, or rewards program from the electronic wallet to the order 1035.

In another embodiment, if a payment processing program, such as Apple Pay or Google Pay, is present natively on the user's device then user follows the pay workflow dictated by their app. If user's device lacks such a payments app, user is presented with a payment detail input page in a web browser, so that the user can enter his or her credit card information. The web payment is processed to determine if payment is accepted.

Upon successful payment 452, a confirmation response 470 can be sent in a reply to the user. The ordering app can then initiate a communication to the client store with the order information that is to be filled by the store for the customer identified by the phone number of the user. In embodiments, the ordering app may forward the order to the client establishment associated with the destination phone number. The order may be printed at an IP enabled printer in the store 480. In an embodiment, the ordering app may store a receipt or other record of the order in the electronic wallet associated with the phone number used in the text messaging.

The intelligent interface 100 can provide additional services on top of the intents requested by a user. For example, a client may wish to make targeted offers to users engaged in text communications with the intelligent interface. A targeted offer program is illustrated in flow chart format in FIG. 6. For a client who has signed up for the targeted offer program, the response generator 116 will call the targeted offer program upon learning of the intent identification from the intent classifier. A decision block 450 determines if a targeted offer is applicable to the intent. Some intents may not qualify for targeted offers. Other intents certainly do qualify for a targeted offer. Still other intents might qualify depending upon object classification. In this latter case, decision block 450 awaits object classification before finally determining whether a targeted offer should be retrieved and sent. If the identified intent does not qualify for a targeted offer 454, the response generator 116 returns to the program for the original intent 455. The program for the intent can then proceed to send 460 its response back to the user in a reply. In some embodiments, the targeted offer 454 may be placed in the electronic wallet associated with the user's phone number for use in later communications with the intelligent interface 100.

If the identified intent does qualify 452, the offer applicable to the intent and/or object is retrieved 456 from among the stored app data 128. A predefined response is retrieved for use in a reply 458. The flow returns to the program for the original intent 455. According to the needs of the system and/or the client, the predefined response for the offer may be sent out ahead of the intent reply, sent out along with the intent reply or sent after the intent reply. The program for the intent can handle the send accordingly 460.

The intelligent texting sessions described above provide business establishments with a valuable electronically-controlled connection to customers. But this connection only gets used if the customer sends the initial text message. Advertising and marketing expenses are generally required to alert customers to the available texting service. To overcome this drawback, the inventors have developed a voice call conversion system that provides an electronically controlled glide path from a customer making a voice phone call to an intelligent texting session with that customer.

As shown in FIG. 1, the voice call conversion system includes a telephony server 500 and the application server 100 loaded with an interactive voice response (IVR) app 600. The telephony server 500 is connected to a telephone network 504 for intercepting telephone calls to any of the client establishments of the overall system. The telephone network 504 may be one or both of a PSTN and SIP network.

The telephony server 500 performs a number of functions through a variety of application modules. A calling line ID interface 510 is used to identify the source of an incoming telephone call. It produces a phone number and can sometimes provide additional information concerning the identity of the person or business associated with the source phone number. A text-to-speech module 520 is a program for generating voice speech. In particular, words of a text can be articulated verbally and transmitted over the phone line to the caller. A speech-to-text module 530 is used to review the voice signals arriving over the telephone network and converting the speech to a text format. A DTMF decoder module 540 identifies the tones produced by activating keys on a telephone keypad. The DTMF decoder module 540 thus identifies keys pressed by a caller. A call control module 550 is in communication with each of the aforementioned modules in the telephony server 500. Upon the telephony server 500 receiving a call, the call control module generates and sends a request 710 over the IP network to the application server 100. The request is directed to the IVR app 600. The request includes the source phone number determined by the calling line ID interface and a client identifier corresponding to the client establishment associated with the phone number that was called by the caller. The request initiates an IVR session in which the IVR app communicates with the call control module 550 and the telephony server 500 communicates with the caller. As the caller responds to voice speech generated by the text-to-speech module 520, the call control module sends subsequent requests to the application server. The subsequent requests may include message content determined by the speech-to-text module 530 or the DTMF decoder 540.

The IVR app 600 interacts with the call control module 550 to conduct an interactive voice response session. A sample flow chart for an interactive voice response session is shown in FIG. 7. A call directed to one of the client establishments over the telephone network 504 is received 700 at the telephony server 500. The call control module 550 issues a request to the IVR app 600. The request includes the source phone number and a client identifier. Based on the client identifier, a suitable IVR session is initiated. Text for an IVR greeting is sent to the call control module, which sends it to the text-to-speech module 520. Thus, a voice greeting 720 is announced over the telephone in the voice call. For a client that is most interested in accepting takeout or delivery orders, the greeting might say something like “Hi, thanks for calling Joe's Pizza! Did you know that you can now skip the wait by sending a text order directly to our phone number? Press or say 1 to start a text order now . . . or press or say 2 to be connected to Joe's!” If the client is a sit-down restaurant, it may be more interested in taking reservations. For such a client, the greeting 720 may say “Hi, thanks for calling Local Business! Did you know that you can now set a reservation by sending a text directly to our phone number? Press or say 1 to start a reservation now . . . or press or say 2 to be connected to the Local Business!” In accordance with alternate embodiments, the telephony server 500 may include digital recordings of the desired greeting 720 for a client. Rather than receiving the text of a greeting from the IVR app 600 and rather than making use of a text-to-speech module, the IVR app can simply identify the recording and the telephony server can play it on the call over the telephone network with the caller.

The caller's response is determined by the speech-to-text module 530, if verbal, or by the DTMF decoder module 540, if push button. If the caller chooses to speak with the client establishment 730, the call is routed 740 to its phone number through PSTN or SIP. When the phone is answered, the caller can engage in a voice call with the client establishment.

If the caller accepts the option to conduct a texting session 750, the telephony server 500 sends a request 760 for a texting session to the application server 100. To the extent the application server has not already received it, or optionally even if the information has been previously sent, the request could include a phone number associated with the caller and a client identifier. The phone number associated with the caller will typically be the source phone number. An option could be implemented for callers from a phone lacking texting capability. According to this option, the IVR session can be further extended to ask for the phone number of the caller's texting device.

In response to the texting session request, the application server implements one of its intent workflows 770. For a client establishment that deals in takeout and deliveries, the texting session may begin by sending message content saying “Here is our menu. What would you like to order?” along with a URI that accesses the establishment's menu. The texting session may continue in similar fashion to the intent explained above with respect to FIG. 5. For a client establishment more interested in taking reservations, the texting session may begin by sending message content saying “When would you like to set the reservation for?” The intent may go on to ask for how many guests would be arriving and for any special requests. Upon completing an order or a reservation, the application server 100 sends the completed item to the client via the IP network. At the end of the texting session, the application server can generate message content for a text to inform the caller that next time he or she can skip making a voice call and text the client establishment directly. The voice call conversion system has thus enabled the caller to experience a texting session and can encourage the caller to make use of this mode of doing business in the future.

The computer implemented method as performed at the application server, shall now be summarized with reference to FIG. 8. After the telephony server receives a voice call over a telephone network 504 to a destination phone number associated with a client, the application server 100 receives a request for an interactive voice response (IVR) session 800 over the IP network from the telephony server 500. The IVR app 600 provides instructions over the IP network to the telephony server to produce a voice inquiry 810 over the telephone network. The IVR voice inquiry offers the caller an option to request a texting session. When the request from the telephony server includes a client identifier, the IVR app 600 can select an IVR workflow with a voice inquiry suitable for the client establishment associated with the client identifier. When a request for a texting session is received 820, an intent workflow program is followed by the application server 100 and therefore message content is generated 830 according to the texting session program associated with the intent. The intent may be selected based upon the client identifier. As described above for the intelligent interface implemented by the application server 100, intents may correspond to programs leading to any of a variety of operations. The texting session will be conducted with a phone number associated with the caller. This will usually be the source phone number identified by the calling line ID interface 510. Alternatively, the caller can be asked to key in or voice a phone number of the texting device the caller intends to use. In all cases, the phone number to be used by the caller for texting is provided to the application server by the telephony server.

The message content generated by the application server, typically in the response generator 116, is packaged into a reply and transmitted with the necessary components of a text message 840. The text message components typically include the source phone number which will be used as the text message destination, the destination phone number that had been dialed by the caller which will be used as the text message source and the generated message content. Texting between the caller and the intelligent interface of the application server may thereafter continue as guided by the intent workflow program. The voice call which was originally intercepted by the telephony server has thus been converted to a texting session. In accordance with aspects of the invention, the destination phone may be a client's landline phone number. Thus, a caller to the landline is introduced to the texting option by the voice call conversion system. Of course, the voice call conversion system works as well when the voice call is to a cellphone number.

Considering the system as a whole, the operation of the voice call conversion system is shown in FIG. 9. A voice call made over the telephone network 504 is intercepted 900 by the telephony server 500. The voice call conversion system is operative when the voice call has been made to a destination phone number corresponding to one of the clients of the application server. The destination phone number may be a cellphone number or a landline number associated with the client.

The system needs to include one or more methods for identifying 910 the source phone number associated with the caller. One approach is to make user of a calling line ID interface 510 in the telephony server 500. To accommodate instances when the calling line ID interface 510 fails to identify the source phone number, the IVR app 600 can include instructions for voice inquiries to the caller requesting a callback number that can receive a text message. The number entered by the caller is detected by the speech-to-text module 530 if it is spoken or the DTMF decoder module 540 if it is entered by push button. The callback number inquiry in the IVR app 600 can also be triggered if the source phone number is a landline, without text messaging capability.

Upon receiving a voice call, the telephony server sends a request 920 over the IP network to the application server requesting initiation of an interactive voice response (IVR) session with the caller. In response to an instruction from the IVR app 600 of the application server 100, the telephony server 500 produces a voice inquiry 930. The voice inquiry may be generated by any suitable method including using a text-to-speech module 520 or by playing a digital recording. At least one of the voice inquiries produced during the IVR session includes an offer to the caller of an option to request a texting session.

When a caller responds affirmatively to the option to request a texting session, the telephony server 500 receives 940 that response requesting a texting session. The response may be verbal or by push button or by using of any other available input mechanism at the caller's phone. The texting session will be conducted with a phone number associated with the caller. The phone number may be the number identified by the calling line ID interface 510 or a number otherwise entered by the caller.

The telephony server 500 sends over the IP network to the application server a request 950 for a texting session to be conducted with the phone number associated with the caller. The request will also be recognized as corresponding to a specific one of the clients. The client identifier may be included in the request or the application server may rely on the client identifier already provided when the IVR session was first requested.

The application server 100 receives the request for a texting session and follows an intent workflow program. Therefore, message content is generated 830 according to the texting session program associated with the intent as described above in connection with FIG. 8. The intent may be selected based upon the client identifier and/or any instruction in the request. The applicable texting session will be conducted with the phone number associated with the caller that was determined in step 910. The phone number to be used for texting the caller will have been provided to the application server by the telephony server in one or more of its requests.

The message content generated by the application server, typically in the response generator 116, is packaged into a reply and transmitted with the necessary components of a text message 840 as described above with regard to FIG. 8. The reply is received at gateway 104 and forwarded 980 by the gateway 104 to the identified phone number of the caller as a text message. The gateway 104 converts the reply, which is an IP network communication, to a text message for communication over a cellular network. A text message sent in response by the caller is intercepted by the gateway 104 on the cellular network. The gateway 104 converts the text message to a request to the application server and sends it over the IP network. Texting between the caller and the intelligent interface of the application server may thereafter continue as guided by the intent workflow program. The voice call which was originally intercepted by the telephony server has thus been converted to a texting session. In accordance with aspects of the invention, the destination phone may be a client's landline phone number. Thus, a caller to the landline is introduced to the texting option by the voice call conversion system. Of course, the voice call conversion system works as well when the voice call is to a cellphone number.

Various embodiments of the invention may be implemented at least in part in any conventional computer programming language. For example, some embodiments may be implemented in a procedural programming language (e.g., “C”), or in an object oriented programming language (e.g., “C++”). Other embodiments of the invention may be implemented as a pre-configured, stand-alone hardware element and/or as preprogrammed hardware elements (e.g., application specific integrated circuits, FPGAs, and digital signal processors), or other related components.

In an alternative embodiment, the disclosed apparatus and methods (e.g., see the various flow charts described above) may be implemented as a computer program product for use with a computer system. Such implementation may include a series of computer instructions fixed either on a tangible, non-transitory medium, such as a computer readable medium (e.g., a diskette, CD-ROM, ROM, or fixed disk). The series of computer instructions can embody all or part of the functionality previously described herein with respect to the system.

Those skilled in the art should appreciate that such computer instructions can be written in a number of programming languages for use with many computer architectures or operating systems. Furthermore, such instructions may be stored in any memory device, such as semiconductor, magnetic, optical or other memory devices, and may be transmitted using any communications technology, such as optical, infrared, microwave, or other transmission technologies.

Among other ways, such a computer program product may be distributed as a removable medium with accompanying printed or electronic documentation (e.g., shrink wrapped software), preloaded with a computer system (e.g., on system ROM or fixed disk), or distributed from a server or electronic bulletin board over the network (e.g., the Internet or World Wide Web). Of course, some embodiments of the invention may be implemented as a combination of both software (e.g., a computer program product) and hardware. The embodiments of the invention described above are intended to be merely exemplary; numerous variations and modifications will be apparent to those skilled in the art. All such variations and modifications are intended to be within the scope of the present invention as defined in any appended claims.

Number	Name	Date	Kind
7720784	Froloff	May 2010	B1
8254535	Madhavapeddi et al.	Aug 2012	B1
8255291	Nair	Aug 2012	B1
8407212	Tzvi et al.	Mar 2013	B2
8718239	Madhavapeddi et al.	May 2014	B1
8775260	Pienkos	Jul 2014	B1
8954349	Tolcher	Feb 2015	B2
9020107	Madhavapeddi et al.	Apr 2015	B2
9143881	Fan et al.	Sep 2015	B2
9147212	Pettyjohn et al.	Sep 2015	B2
9185215	Madhavapeddi et al.	Nov 2015	B2
9189742	London	Nov 2015	B2
9251508	Bishop	Feb 2016	B2
9271129	Lew et al.	Feb 2016	B2
9420437	Johnston et al.	Aug 2016	B2
9465864	Hu et al.	Oct 2016	B2
9553997	Sharma et al.	Jan 2017	B2
9596350	Dymshyts et al.	Mar 2017	B1
9626446	Burris	Apr 2017	B2
9674372	Karnas et al.	Jun 2017	B2
9754176	Kostyukov	Sep 2017	B2
9779356	Driscoll	Oct 2017	B2
20080071544	Beaufays et al.	Mar 2008	A1
20120290609	Britt	Nov 2012	A1
20130013499	Kalgi	Jan 2013	A1
20130165086	Doulton	Jun 2013	A1
20140280267	Perlegos	Sep 2014	A1
20150234636	Barnes, Jr.	Aug 2015	A1
20150281166	Woo et al.	Oct 2015	A1
20170161393	Oh et al.	Jun 2017	A1
20180336266	Chen et al.	Nov 2018	A1
20210256507	Kushevsky	Aug 2021	A1

	Number	Date	Country
Parent	16522177	Jul 2019	US
Child	16752344		US
Parent	15947084	Apr 2018	US
Child	16522177		US

Apparatus and method for payment of a texting session order from an electronic wallet

Information

Patent Number

Date Filed

Date Issued

Inventors

Original Assignees

Examiners

Agents

CPC

Field of Search

CPC

International Classifications

Term Extension

Abstract

Description

Claims

Priority Claims (1)

CROSS REFERENCE TO RELATED APPLICATIONS

US Referenced Citations (32)

Foreign Referenced Citations (1)

Non-Patent Literature Citations (2)

Related Publications (1)

Continuation in Parts (2)

Entry
Susanto et al., “Factors Influencing Citizen Adoption of SMS-Based e-Govemment Services”, Electronic Journal of e-Government, vol. 8, Issue 1, pp. 55-71, Mar. 2010.
“Text Appointments Overview”, TimeTrade, https://www.timetrade.com/resource/interactive-text-demo/, 5 pages, 2018.