Patent Application
20250220297
This disclosure relates generally to a computer vision-based self-service checkout terminal, and more particularly to a system and method for calibrating camera position in such computer vision-based self-service checkout terminal.
A vision system for use with a self-service checkout terminal is a technology that incorporates computer vision and image processing techniques to enable automated product recognition, item tracking, and transaction handling in a self-checkout environment. The vision system uses cameras and specialized software to capture image data of items and analyze and interpret the captured image data to, for example, identify the items. The vision system interfaces with software in the self-checkout terminal to convey the recognized items, quantities, and prices. The vision system is coupled via an interface to the point-of-sale (POS) system in the self-service checkout terminal and automates the transaction process by sending the item details for pricing, discounts, and payment processing. By leveraging a vision system, self-service checkout terminals provide a more efficient and streamlined experience for customers, reduce reliance on manual intervention, minimize errors, and improve overall accuracy and convenience during the checkout process.
Traditional self-service checkout terminals require a barcode on each item to be scanned separately. Vision systems speed up the checkout process by processing groups of items at the same time. However, customers must place all of the items to be purchased into the field of view of the several cameras in the vision system (“the scan zone”) and remove any obstructions to viewing such items. Each of the cameras in the vision system must be positioned at a precise location and orientation in space in order to provide an accurate view of the scan zone (each camera will have a pre-assigned “target view” of the scan zone). Because the self-checkout system consists of hardware components assembled from materials (e.g., sheet metal and plastic) with inherent tolerances, the positioning of each camera in the system may require adjustment from time to time (e.g., at initial assembly, at installation, after repositioning of the system, etc.) to ensure that such camera provides the required target view. Acrylic templates have been used for camera calibration that are partly transparent in order to show the expected scan zone for the associated camera. This requires a separate acrylic template for each camera that includes an outline imposed thereon of the scan zone view for that particular camera (i.e., the acrylic template outlines or otherwise identifies the position of the scan zone within the field of view of the camera). The acrylic templates are sequentially placed over a display for the self-service checkout terminals (or a display for a computer system coupled thereto), allowing a technician to either confirm that each camera is properly targeted at the scan zone or adjust each camera to be properly targeted. This requires separate acrylic templates for each camera (because each camera will have a different view of the scan zone). Using such acrylic templates can be unwieldy and requires sets of acrylic templates for each system (or at least each store location).
The present disclosure describes a technical solution that solves the above-noted technical problems.
The following detailed description, given by way of example and not intended to limit the present disclosure solely thereto, will best be understood in conjunction with the accompanying drawings in which:
In the present disclosure, like reference numbers refer to like elements throughout the drawings, which illustrate various exemplary embodiments of the present disclosure.
Referring now to
The network switch/hub 130 is coupled to a processing portion 140 of the computing section 110 of system 100. The processing portion 140 includes a processor 142 and a memory 146. Memory 146 includes both volatile (RAM) portions and nonvolatile (non-transitory computer readable storage medium) portions. System 100 may include more than one processing portion, e.g., one portion for processing the camera signals and performing analysis thereof, and another portion for operating the checkout functions of the terminal. The processing portion 140 is coupled to a user interface 160 that includes, inter alia, a display 162 and a keyboard 164 (or other type of data entry device). The user interface 160 is used during normal operations of the system 100 and may also be used by a technician with appropriate access rights to access a setup menu for installation and configuration of system 100, including as discussed below with respect to the flowchart of
Computing section 110 also includes a network interface 150 coupled to processing portion 140 and further coupled to an internal network 180 at a retail store site. An external (local) computer 170 may be coupled to the network 180 via a network interface (not shown). The external computer 170 includes, inter alia, a processor 176, a memory 178, a display 172, and a keyboard (or other user input device) 174. Memory 174 includes both volatile (RAM) portions and nonvolatile (non-transitory computer readable storage medium) portions. A technician can also access the setup menu for installation and configuration of system 100 via local computer 170 using, for example, remote login software as known in the art. The remote login software consists of commands stored in the nonvolatile portion of memory 178 and executed by processor 176.
Referring now to
The setup menu provides a technician with access to a tool for use in calibrating the position of cameras 120, 122, 124, 126 and evaluating the proper operation of such cameras, either on-site or remotely. Each of the cameras 120, 122, 124, 126 is preset for a particular depth of field and diagonal field of view. When the technician performs the camera calibration, each camera is adjusted to so that the scan zone (or an outline thereof provided on the surface of or in proximity to the scan zone) matches as close as possible the position of the expected scan zone marked on an overlay combined with the video signal. In some cases, it will not be possible to perfectly match the actual scan zone or the outline thereof to the overlay scan zone because the camera is out of specification for either depth of field or diagonal field of view. When this happens, if the depth of field or diagonal field of view settings are too far out of specification, the camera can be replaced (e.g., when these camera settings are fixed) or the camera settings can be adjusted to be within specification (when the camera provides the ability to adjust such settings).
Referring now to
By including the calibration tool in system 100, camera calibration is easily performed without any need for separate acrylic templates for each camera. This both reduces cost and makes setup much easier. This ensures that system 100 can be maintained in proper calibration and therefore provides consistent and accurate operations.
Although the present disclosure has been particularly shown and described with reference to the preferred embodiments and various aspects thereof, it will be appreciated by those of ordinary skill in the art that various changes and modifications may be made without departing from the spirit and scope of the disclosure. It is intended that the appended claims be interpreted as including the embodiments described herein, the alternatives mentioned above, and all equivalents thereto.
