Patent Application
20230046686
The present disclosure relates to systems and methods for providing a frictionless commerce experience to customers.
Frictionless commerce is currently a popular trend permeating many facets of today's customer shopping experience. In general, frictionless commerce leverages technology to improve a customer's retail experience and to increase sales by recommending certain products for purchase to the customer. To accomplish its goal, frictionless commerce uses data stored on the customer's devices (e.g., Smartphones), as well as software applications executing on those devices and/or on websites, to seamlessly integrate purchasing activities and purchasing opportunities into the customer's shopping experience.
Embodiments of the present disclosure provide shoppers with a frictionless commerce experience. More particularly, a computer system configured according to the present embodiments determines, tracks, and stores certain physical characteristics and/or attributes of the products selected by a shopper for purchase. The system then analyzes these attributes with respect to the shopper's particular purchasing habits and uses the results of that analysis to make product recommendations to the shopper. The recommendations may be for the same preferred product or, if that product is out-of-stock, for a similar product.
As an example, consider situations where a shopper purchases produce at a retail store. Over time, a system configured according to the present embodiments captures information associated with the physical attributes of the selected produce, such as the ripeness and/or freshness of the produce. As described in more detail later, such information may be captured, for example, using cameras positioned throughout the retail store and/or provided by the shopper. The system can then analyze the captured information to determine whether the shopper prefers to purchase particular produce at a particular stage of ripeness and/or freshness. If so, the system can send messages to the shopper's mobile device recommending that the shopper purchase produce at the determined stage of ripeness and/or freshness during the shopper's subsequent visits.
As defined herein, a “shopper” is an entity that selects products and/or services offered by a retail establishment for purchase by the general public. For example, a shopper may be a customer of the retail establishment. Additionally, or alternatively, a shopper may be a “picker” (i.e., an employee of the retail establishment whose job it is to select the products and/or services for purchase by a customer of the retail establishment). In these latter cases, the recommendation messages provided to a device of a picker are generated based on the preferences of the customer for which he/she is shopping. Thus, when the picker is selecting produce for a given customer, the system would send messages to the picker's device recommending that the picker select produce at the given customer's preferred stage of ripeness and/or freshness.
Regardless of the particular products, services, or whether the shopper is a customer or a picker, however, a system configured according to the present embodiments provides customized purchase recommendations to the shopper based on information that is repeatedly updated and analyzed over time. This allows the system to remain “up-to-date” with respect to a given customer's changing preferences and to provide recommendations that are most appropriate for a shopper.
Turning now to the drawings,
As seen in
In this embodiment, cameras 20 comprises a plurality of cameras configured to capture images of the products a shopper selects for purchase. As is known in the art, cameras 20 may be distributed throughout the store and be positioned to capture images at different angles. The images may be utilized to identify the particular products selected for purchase by a shopper, but also to identify the shopper using known facial recognition techniques. Thus, one or more of the cameras 22, 24, 26, 28 may be positioned at the entrance/exit of the retail establishment, and/or along the aisles of the retail establishment, and/or at the checkout stations of the retail establishment. Regardless of their positioning, however, the images captured by the cameras 20 are communicated to computing device 90 for image processing, analysis, and storage in DB 100.
Computing device 90 may be any computing device known in the art and, as described in more detail below, comprises processing circuitry and software instructions configured to implement embodiments of the present disclosure. By way of example only, computing device 90 may be an application server (AS) configured to operate according to the present disclosure. Regardless of its particular structure, however, computing device 90 is configured according to the present embodiments to obtain digital images of a product selected for purchase by a shopper, perform a digital image analysis on those digital images, and then determine a state of the product based on the analysis. Based on the results of that analysis and on one or more predefined preferences regarding a customer's preferred state of a product, computing device 90 is configured to generate feedback information for transmission to the shopper in one or more messages. As stated above, these messages include information recommending a product having the same or similar state as that preferred by the customer.
Method 50 is implemented, in one embodiment, by computing device 90 and begins with obtaining one or more digital images of a product selected by a shopper for purchase for a customer order (box 52). The images may be captured by one or more of the cameras 20 at one or more different angles throughout the retail establishment. A state of the product is then determined based on a digital analysis performed on the captured images (box 54). For example, the analysis may identify a coloring or shade of a particular piece of produce, such as the green color of a banana peel, and based on that information, determine that the banana is not yet ripe. In another example, the analysis may identify a “best if used by” or “expiration” date on a label of a product. Based on that date information, the analysis may determine that the product is no longer fresh.
Regardless, in one embodiment, the results of the analysis may cause computing device 90 to modify the sales price of the product (box 56). Such is beneficial for a variety of reasons. Specifically, products that are determined to be in a particular state can be “marked-down” in price making them more appealing to the customer for purchase. Of course, the opposite is also true. That is, products determined to be in a particular state can be “marked-up” in price, or at least not discounted, based on the knowledge that the customer selecting these products actually prefers them to be in the determined state.
However, regardless of whether the price of the product is marked-up, discounted, or left alone, method 50 configures computing device 90 to generate feedback information based on the state of the product and on one or more of the customer preferences for the state of the product (box 58), and transmitted in a message to the customer's device (box 60). The feedback information may, as described previously, recommend that the customer purchase a particular product at the stage of ripeness and/or freshness deemed desirable to the customer. Method 50 then updates the predefined customer preferences based on the determined state of the selected product (box 62). As previously described, customer preferences may change over time; however, the present embodiments track those changing preferences thereby allowing system 10 to maintain a current database of customer preference information.
As stated above, computing device 90 is configured to determine various characteristics and/or attributes of a product based on the digital image analysis performed on the captured digital images.
As seen in
Any method known in the art may be utilized to optically identify the color of a piece of produce. For example, in one embodiment, an operator, such as a retail store employee, defines one or more stages of ripeness for each piece of produce offered for sale by the retail establishment. The number of stages, in one embodiment, is arbitrary. The operator then associates each stage with a particular color. According to the present embodiments, computing device 90 determines the color of the produce based on the results of the digital image analysis and compares that color to the colors associated with the different pre-defined stages of ripeness. Based on the results of that color comparison, computing device 90 can identify the particular stage of ripeness for the product.
In one embodiment, a table structure, such as shown below in Table 1, is stored in memory mapping the different ripeness stages to a corresponding color or shade of color. For illustrative purposes, the different colors are indicated using corresponding indicative labels “A,” “A/B,” . . . etc.
In another embodiment, the operator associates a plurality of ripeness stages for each piece of produce to a spectral reflectance or transmittance value for the product. In these embodiments, computing device 90 utilizes known techniques to estimate the spectral reflectance or transmittance value for the produce based on digitally processing the captured images. Once a value indicating the reflectance/transmittance is determined (which in this embodiment is defined as a percentage), computing device 90 compares that value to the pre-defined values provisioned by the operator and identifies the ripeness stage of produce based on the results of that comparison. In a variant of this embodiment, the resultant value can be translated or mapped to a corresponding color, which is then used to determine the corresponding ripeness stage. Table 2 below illustrates the type of information that can be stored in a table according to one embodiment.
In addition to, or in lieu of a ripeness stage, computing device 90 is also configured to determine the various characteristics and/or attributes of a product based the “freshness” of the product.
In the embodiment of
As seen in Table 3, each Life Cycle Stage is associated with a scalar “Date Differential” value. As stated above, these values indicate the number of days separating the expiration or use-by date on the product label from the current date. For example, a “−7” would indicate that the date printed on the label is still 7 days away from the date corresponding to that of the 1st life cycle stage, and therefore, considered very fresh. Similarly, a “−5” would indicate that the current date is still 5 days away from the date corresponding to that of the 2nd subsequent life cycle stage, but still considered fresh. A “+2” would indicate that the current date is 2 days beyond the determined expiration or use-by date, and therefore, considered to be not fresh. Regardless of the particular values used, however, the information in Table 3 is used by computing device 90 to identify the how fresh the product selected by a shopper is.
Those of ordinary skill in the art should appreciate that the present embodiments also configure computing device 90 to identify label information other than an expiration date or use-by date. Such other information includes, but is not limited to, the Stock Keeping Unit (SKU) printed on a label. Based on the SKU, computing device 90 can determine, using information stored in DB 100, for example, when the particular product was placed on a shelf or display area for purchase, the lot number associated with the product, the expiration date, the use-by date, and/or other data by which computing device 90 can determine the ripeness and/or freshness of the product.
Determining the ripeness and/or freshness of a product, and tracking the personal shopping/selection habits of customers according to the present embodiments in general, provides benefits to both consumers and the owners of the retail establishments that conventional systems do not provide. For example, a system 10 configured according to the present embodiments incentivizes consumers to purchase certain recommended products. This is because the products recommended by computing device 90 have the same or similar physical characteristics and/or attributes as those previously purchased by the consumer. Therefore, the shopper is more inclined to purchase the recommended products.
Further, determining the ripeness and/or freshness of a product allows the owners of the retail establishments to dynamically adjust the prices for certain products. For example, products that have a limited shelf life (e.g., dairy products, vegetables, fruit, etc.) and that are at a later ripeness stage or life cycle stage could be offered at a larger discount to a shopper. Not only does the lower price incentivize the shopper to purchase produce having a shorter shelf life but it also reduces food waste. Selling more product while simultaneously reducing waste, in turn, results in a higher volume of sales and increased income for the retail establishment.
Additionally, with conventional methods of frictionless shopping, a picker will arbitrarily select a similar replacement product whenever the customer's desired product is out of stock in which case, the customer may end up paying for a replacement product that he/she does not want. Alternatively, the picker will contact the customer requesting them to identify a satisfactory replacement product. In these latter cases, the customer's frictionless shopping experience is hindered by the messages from the picker. With a computing device 90 configured according to the present embodiments, however, the customer preferences are readily available allowing the picker to select a desired product, or a replacement product, more accurately.
Those of ordinary skill in the art should readily appreciate that the present embodiments are not limited solely to determining and utilizing the measured freshness and/or ripeness of a product. Rather, other embodiments of the present disclosure glean and utilize other information from the digital analysis of the captured images. For example, in one embodiment, computing device 90 is configured to determine the number of products on a shelf or display area whenever a shopper selects that product. Based on this number, computing device 90 can determine whether the selected product was only recently placed on the shelf or display area, indicating that it is “fresh,” or whether it has been on the shelf or display area for some period of time.
In another embodiment, cameras 20 can capture images showing the position of the product on a shelf or display area. In these cases, computing device 90 may determine that products positioned farther back on a shelf or display area are “older” than those positioned closer to the front of the shelf or display area. Alternatively, computing device 90 may determine that products positioned more towards the front of the are “older” than those positioned closer to the back of the shelf or display area. Regardless, computing device 90 can dynamically modify the price of the product based on the determination, as described above, and send the modified price to the shopper's device in a transmitted message thereby incentivizing the shopper to purchase the product and allowing the retail establishment owner to “move” its merchandise and reduce waste.
In another embodiment, retail establishments label their products with temperature-sensitive labels. As is known in the art, such labels change color based on the temperature of the product. In these embodiments, at least one of the cameras 20 would capture the temperature indication of the label. So long as the temperature of the product (e.g., meat, dairy, etc.) remains within a predetermined temperature range, the color of the label remains constant. However, when the temperature of the product exceeds or dips below the predetermined range, the label changes color. Using this information, cameras 20 capture the product label in the digital image sent to computing device 90. Then, as previously described, computing device 90 determines the color of the product label and adjusts the price accordingly. Additionally, computing device 90 will update the customer preferences with respect to the product selection and use the updated information to make future product recommendations to the shopper as previously described.
As seen in
The communications interface circuitry 94 comprises, in one embodiment, a transceiver circuit and/or interface circuit for communicating with remote devices and systems, such as routers 14, communications system 16, the shopper's personal wireless device 18, cameras 20, and DB 100. For example, using communications interface circuitry 94, computing device 90 can receive, as previously described digital images of the products selected by the shopper from the one or more cameras 20 distributed throughout the retail establishment, data and information from DB 100, including but not limited to copies of the digital images, results of previously performed image analyses performed on the digital images, and the customer preferences. In this regard, the communications interface circuitry 94 according to embodiments of the present disclosure may comprise one or more of a WiFi interface, a cellular radio interface, a BLUETOOTH interface, an Ethernet interface, or other similar interface for communicating over a communication network or a wireless communication link.
Memory circuitry 96 comprises a non-transitory computer readable medium that stores executable program code and data used by the processing circuitry 92 for operation. In this embodiment, the program code and data comprises a control program 98 that, when executed by processing circuitry 92, configures computing device 90 to perform the functions previously described. In some embodiments, control program 98 has access to customer preference information that, as previously described, can be utilized to recommend products for a shopper to purchase. Memory circuitry 96 may include both volatile and non-volatile memory, and may comprise random access memory (RAM), read-only memory (ROM), and electrically erasable programmable ROM (EEPROM) and/or flash memory. Additionally, or alternatively, memory circuitry 96 may comprise discrete memory devices, or be integrated with one or more microprocessors in the processing circuitry 92.
The digital image obtaining unit/module 110 comprises program code that is executed by processing circuitry 92 to obtain the digital images of a shopper and of the products selected by the shopper. Such images may be obtained from DB 100, from one or more of the cameras 20, and/or from one or more other computing devices via network 12.
The product state determination unit/module 112 comprises program code executed by processing circuitry 92 to determine the state of the products selected by the shopper for purchase. For example, as described above, the product state determination unit/module 112 can configure the processing circuitry 92 to determine the stage of ripeness and/or freshness of a product, or any other state of a product based on an analysis of the product's physical characteristics and/or attributes. To this end, the product state determination unit/module 112 can configure the processing circuitry 92 to perform the analysis based on the digital images obtained by the digital image obtaining unit/module 110, and/or analyses performed by other computing devices.
The order modification unit/module 114 comprises program code executed by processing circuitry 92 to modify the pricing of a product selected by the shopper, as previously described, based on the determinations made by the product state determination unit/module 112. By way of example only, the order modification unit/module 114 may control processing circuitry 92 to discount the current price for produce that is deemed to be in a later stage of freshness or ripeness.
The feedback generation unit/module 116 comprises program code executed by processing circuitry 92 to recommend certain products to the shopper for purchase. For example, in one embodiment, the feedback generation unit module 116 generates a message indicating the results of the product state determination unit/module 112 and/or the order modification unit/module 114. These results recommend a product having the same or similar physical characteristics and/or attributes as the products historically selected for purchase by the shopper. The results can also indicate a change in price for the recommended product. So generated, the feedback generation unit module 116 sends the message to the shopper's personal device (e.g., wireless device 18) for display to the shopper.
The communications interface unit/module 118 comprises program code executed by processing circuitry 92 to facilitate communicating data and information with one or more remote devices via one or more communications networks. Such devices include, but are not limited to, routers 14, communications system 16, the shopper's personal wireless device 18, cameras 20, and DB 100. As described above, such data and information includes, but is not limited to, the digital images obtained by cameras 20, customer preference information tracked and stored in DB 100, the results of digital image analyses performed on the digital images obtained by cameras 20, and the messages generated by the feedback generation unit/module 116 for transmission to the shopper's personal wireless device 18.
The present embodiments may, of course, be carried out in other ways than those specifically set forth herein without departing from essential characteristics of the invention. For example, the embodiments previously described utilize digital imagery captured by one or more cameras 20 positioned throughout a retail establishment to determine the state of a product purchased by a shopper. However, those of ordinary skill in the art will readily appreciate that this is merely for illustrative purposes. In one embodiment, system 10 may also capture and utilize information and data captured by other devices in addition to, or in lieu of, cameras 20. As seen in
Therefore, the present embodiments are to be considered in all respects as illustrative and not restrictive, and all changes coming within the meaning and equivalency range of the appended claims are intended to be embraced therein.
