This invention relates generally to self-checkout verification.
Generally, after a customer pays for the purchased items at a retail facility, the customer will have to show a purchased receipt to an associate before leaving the retail facility in order for the associate to verify that the items in the customer’s cart or in the customer’s possession have been paid. However, this may result in assigning some of the associates to perform this task when the associates time can be better utilized elsewhere in the retail facility. Additionally, there may result in unnecessary long customer lines just to leave the retail facility.
Disclosed herein are embodiments of systems, apparatuses and methods pertaining to self-checkout verification at a retail facility. This description includes drawings, wherein:
Elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions and/or relative positioning of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of various embodiments of the present invention. Also, common but well-understood elements that are useful or necessary in a commercially feasible embodiment are often not depicted in order to facilitate a less obstructed view of these various embodiments of the present invention. Certain actions and/or steps may be described or depicted in a particular order of occurrence while those skilled in the art will understand that such specificity with respect to sequence is not actually required. The terms and expressions used herein have the ordinary technical meaning as is accorded to such terms and expressions by persons skilled in the technical field as set forth above except where different specific meanings have otherwise been set forth herein.
Generally speaking, pursuant to various embodiments, systems, apparatuses and methods are provided herein useful for self-checkout verification at a retail facility. In some embodiments, a system for self-checkout verification at a retail facility includes an optical imaging unit mounted at a location proximate an exit of the retail facility. The optical imaging unit may obtain data from a purchase receipt and/or images of items placed into a container by a customer. The system includes a control circuit communicatively coupled to the optical imaging unit via a communication network. In some embodiments, the control circuit receives purchase receipt data in response to the optical imaging unit scanning a machine-readable identifier of the purchase receipt. In some embodiments, the control circuit receives one or more images of the items in the container captured by the optical imaging unit in response to the scanning of the machine-readable identifier of the purchase receipt. The control circuit executes a machine learning model trained to perform item detection, item classification, and item verification of each item shown in the one or more images to automatically identify the items in the container, and/or output electronic data corresponding to an electronic receipt of the items in the container that were identified by the machine learning model. In some embodiments, the control circuit automatically detects each unpaid item of the items in the container based on a comparison of the purchase receipt data with the electronic data. In some embodiments, the control circuit provides an alert signal in response to automatically detecting an unpaid item.
In some embodiments, a method for self-checkout verification at a retail facility includes obtaining, by an optical imaging unit mounted at a location proximate an exit of the retail facility, data from a purchase receipt and images of items placed into a container by a customer. The method may include receiving, by a control circuit communicatively coupled to the optical imaging unit via a communication network, purchase receipt data in response to the optical imaging unit scanning a machine-readable identifier of the purchase receipt. The method may include receiving, by the control circuit, one or more images of the items in the container captured by the optical imaging unit in response to the scanning of the machine-readable identifier of the purchase receipt. In some embodiments, the method includes executing, by the control circuit, a machine learning model trained to perform item detection, item classification, and item verification of each item shown in the one or more images to automatically identify the items in the container, and/or output electronic data corresponding to an electronic receipt of the items in the container that were identified by the machine learning model. The method may include automatically detecting, by the control circuit, each unpaid item of the items in the container based on a comparison of the purchase receipt data with the electronic data. The method may include providing, by the control circuit, an alert signal in response to automatically detecting an unpaid item.
The present disclosure is a self-serve checkout shrinkage reduction systems and methods that prevent shrinkage in self-checkout terminals at retail facilities and/or exit door areas. The present disclosure is applicable in purchase transactions occurring at retail facilities including at a cashier, scan and go and self-checkout. The present disclosure provides no-touch and self-service for customers.
Additional disclosures are provided in U.S. Application No. 16/931,076 filed Jul. 16, 2020 and PCT Application No. PCT/US20/60120 filed Nov. 12, 2020, all which are incorporated herein by reference in their entirety.
At step 808, the control circuit 102 may execute a machine learning model 114 trained to perform item detection, item classification, and/or item verification of each item shown in the one or more images to automatically identify the items in the container. Further, at step 808, the control circuit 102 may execute the machine learning model 114 trained to output electronic data corresponding to an electronic receipt of the items in the container that were identified by the machine learning model 114. In some embodiments, at step 810, the control circuit 102 automatically detects each unpaid item of the items in the container based on a comparison of the purchase receipt data with the electronic data. In some embodiments, at step 812, the control circuit 102 provides an alert signal in response to automatically detecting an unpaid item. In some embodiments, the machine learning model 114 is stored in a memory 112. In some embodiments, the memory 112 includes hard disk drives, solid state drives, optical storage devices, flash memory devices, random access memory, read only memory, and/or cloud storage devices.
In some embodiments, the machine learning model 114 may be based on a machine learning algorithm including a supervised learning, an unsupervised learning, a reinforcement learning, binary classification, Support Vector Machine (SVM), artificial neural networks, convolutional neural networks, You Only Look Once (YOLO), RetinaNet, Regional based CNN (RCNN), Fast-RCNN, Faster-RCNN, and Mask RCNN, and/or any one or more open-sourced machine learning algorithm available to public for download and use. Those skilled in the art will recognize that the embodiments described herein can use one or more publicly known and/or privately created machine learning algorithm without departing from the scope of the invention. In some embodiments, the machine learning algorithm may be iteratively input a plurality of images of various items in order for the machine learning algorithm to output a machine learning model 114 that is able to and/or trained to automatically identify and/or recognize items generally sold and/or purchased at a retail facility within a predetermined accuracy. In the item detection step, to make sure our model can detect all types of products from different angles, we designed algorithm to create 3D model of representative products and simulated thousands of shopping carts with different product combinations. In the item recognition step, our model not only considers the text information of each product including how large is the text and where it is positioned on the product, the model also considers the packaging features like color and shape of a product. In the verification step, our model can tell or identify whether a captured or a cropped shopping cart image includes a single product or not with high confidence to reduce false positive predictions based on synergy of text, color and shape features. In some embodiments, the control circuit 102 may find or detect all the possible items in a cart (e.g., the container 204) and draw bounding boxes on those found/detected items. By one approach, if there is only one item found/detected, the control circuit 102 may draw one bounding box. By another approach, if there are ten items found/detected, the control circuit 102 may draw ten bounding boxes. In response, for each bounding box, the control circuit 102 may determine what the item found/detected is based on an associated confidence score. In some embodiments, the control circuit 102 may determine the confidence score by comparing text and image features of each item image in the bounding box with stored images of items in a database accessible by the control circuit 102. For example, the database includes training templates of all the UPCs (e.g., images of items with associated UPCs used to train the machine learning model 114). The confidence score may be a combined weighted score based on similarities of text, color and shape features of each found/detected item with a particular item associated with a stored image. In some embodiments, the determined confidence score is compared with a predetermined threshold by the control circuit 102. By one approach, if the determined confidence score is at least equal to the predetermined threshold, the control circuit 102 may determine that the detected/found item is the same item as the particular item associated with the stored image that the detected/found item is compared with.
Further, the circuits, circuitry, systems, devices, processes, methods, techniques, functionality, services, servers, sources and the like described herein may be utilized, implemented and/or run on many different types of devices and/or systems.
By way of example, the system 900 may comprise a processor module (or a control circuit) 912, memory 914, and one or more communication links, paths, buses or the like 918. Some embodiments may include one or more user interfaces 916, and/or one or more internal and/or external power sources or supplies 940. The control circuit 912 can be implemented through one or more processors, microprocessors, central processing unit, logic, local digital storage, firmware, software, and/or other control hardware and/or software, and may be used to execute or assist in executing the steps of the processes, methods, functionality and techniques described herein, and control various communications, decisions, programs, content, listings, services, interfaces, logging, reporting, etc. Further, in some embodiments, the control circuit 912 can be part of control circuitry and/or a control system 910, which may be implemented through one or more processors with access to one or more memory 914 that can store instructions, code and the like that is implemented by the control circuit and/or processors to implement intended functionality. In some applications, the control circuit and/or memory may be distributed over a communications network (e.g., LAN, WAN, Internet) providing distributed and/or redundant processing and functionality. Again, the system 900 may be used to implement one or more of the above or below, or parts of, components, circuits, systems, processes and the like. For example, the system 900 may implement the system for self-checkout verification at a retail facility with the control circuit 102 being the control circuit 912.
The user interface 916 can allow a user to interact with the system 900 and receive information through the system. In some instances, the user interface 916 includes a display 922 and/or one or more user inputs 924, such as buttons, touch screen, track ball, keyboard, mouse, etc., which can be part of or wired or wirelessly coupled with the system 900. Typically, the system 900 further includes one or more communication interfaces, ports, transceivers 920 and the like allowing the system 900 to communicate over a communication bus, a distributed computer and/or communication network (e.g., a local area network (LAN), the Internet, wide area network (WAN), etc.), communication link 918, other networks or communication channels with other devices and/or other such communications or combination of two or more of such communication methods. Further the transceiver 920 can be configured for wired, wireless, optical, fiber optical cable, satellite, or other such communication configurations or combinations of two or more of such communications. Some embodiments include one or more input/output (I/O) interface 934 that allow one or more devices to couple with the system 900. The I/O interface can be substantially any relevant port or combinations of ports, such as but not limited to USB, Ethernet, or other such ports. The I/O interface 934 can be configured to allow wired and/or wireless communication coupling to external components. For example, the I/O interface can provide wired communication and/or wireless communication (e.g., Wi-Fi, Bluetooth, cellular, RF, and/or other such wireless communication), and in some instances may include any known wired and/or wireless interfacing device, circuit and/or connecting device, such as but not limited to one or more transmitters, receivers, transceivers, or combination of two or more of such devices.
In some embodiments, the system may include one or more sensors 926 to provide information to the system and/or sensor information that is communicated to another component, such as the control circuit 102, the first optical imaging unit 104, the second optical imaging unit 106, the third optical imaging unit 108, the display unit 116, the memory 112, etc. The sensors can include substantially any relevant sensor, such as temperature sensors, distance measurement sensors (e.g., optical units, sound/ultrasound units, etc.), optical based scanning sensors to sense and read optical patterns (e.g., bar codes), radio frequency identification (RFID) tag reader sensors capable of reading RFID tags in proximity to the sensor, and other such sensors. The foregoing examples are intended to be illustrative and are not intended to convey an exhaustive listing of all possible sensors. Instead, it will be understood that these teachings will accommodate sensing any of a wide variety of circumstances in a given application setting.
The system 900 comprises an example of a control and/or processor-based system with the control circuit 912. Again, the control circuit 912 can be implemented through one or more processors, controllers, central processing units, logic, software and the like. Further, in some implementations the control circuit 912 may provide multiprocessor functionality.
The memory 914, which can be accessed by the control circuit 912, typically includes one or more processor readable and/or computer readable media accessed by at least the control circuit 912, and can include volatile and/or nonvolatile media, such as RAM, ROM, EEPROM, flash memory and/or other memory technology. Further, the memory 914 is shown as internal to the control system 910; however, the memory 914 can be internal, external or a combination of internal and external memory. Similarly, some or all of the memory 914 can be internal, external or a combination of internal and external memory of the control circuit 912. The external memory can be substantially any relevant memory such as, but not limited to, solid-state storage devices or drives, hard drive, one or more of universal serial bus (USB) stick or drive, flash memory secure digital (SD) card, other memory cards, and other such memory or combinations of two or more of such memory, and some or all of the memory may be distributed at multiple locations over the computer network. The memory 914 can store code, software, executables, scripts, data, content, lists, programming, programs, log or history data, user information, customer information, product information, and the like. While
Those skilled in the art will recognize that a wide variety of other modifications, alterations, and combinations can also be made with respect to the above described embodiments without departing from the scope of the invention, and that such modifications, alterations, and combinations are to be viewed as being within the ambit of the inventive concept.
This application claims the benefit of U.S. Provisional Application No. 63/304,926 filed Jan. 31, 2022, which is incorporated herein by reference in its entirety.
Number | Date | Country | |
---|---|---|---|
63304926 | Jan 2022 | US |