Patent Application
20240330854
The invention relates to a method for logically linking an electronic display unit to a product.
In modern shops product and/or price information is presented on electronic display units, in particular electronic price displays, in technical jargon called electronic shelf labels, ESL for short. Each ESL displays product and/or price information for a corresponding product, i.e. a product in the immediate vicinity of the ESL, on its screen. In order for the ESL to be able to display the correct information for the corresponding product, the ESL must first be logically linked to the correct product, i.e. a digitally stored relationship must be established, which in technical jargon is called matching. Up to now this has been realised in that a staff member with a mobile barcode scanner walks through the aisles of the shop and in each case scans or records with his/her device the barcode depicted on the screen of the ESL and on the packaging of the corresponding product. These barcodes permit unique identification of the ESL and the product. The barcodes recorded by the barcode scanner device are forwarded to a shop management software, where the logical link of the respective ESL to the corresponding product is stored digitally in a data base, the shop management software running on a server or being made available in a cloud-based manner. This method is however very time-consuming and binds personnel resources.
This has led to the invention making it its objective to provide a method, with which these disadvantages are overcome.
This objective is met by a method according to claim 1. The subject of the invention therefore is a method for logically linking an electronic display unit to a product by means of a linking device, wherein the display unit and the product are located on a mounting structure in the detection region of an image detecting device, wherein the method comprises the method steps listed below, namely: generating image data, which represent an optical depiction of the product and the display unit by means of the image detecting device and transmitting these image data to the linking device, by means of which the logical link can be established, generating a trigger signal by means of a trigger stage assigned to the mounting structure, to which the display unit is attached, wherein the trigger signal indicates the presence of an unlinked display unit in the detection region, and logically linking the as yet unlinked electronic display unit to the product in accordance with the occurrence of the trigger signal.
The inventive measures have the advantage that logically linking an electronic display unit to a product is performed fully automatically, whereby the time spent in terms of human resources for establishing such a logical linking is decisively reduced, possibly even completely avoided, and possible human-resource-related errors are completely excluded. This effect, which readily occurs for even a single display unit, is highly noticeable in particular in shops, in which several thousands of such display units have to be installed or managed/maintained in the course of time.
In this regard the logical linking process or the point in time at which logical linking is initiated, is initiated by the actual occurrence of a new display unit. Viewed in a temporal context the trigger signal indicates, when or as from when a logical linking between a display unit and a product has to be carried out.
Taking this measure means that the otherwise required permanent processing of the image data transmitted from the image detecting device to the linking device becomes redundant, whereby it would have to be constantly checked, whether an as yet unlinked display unit has appeared in the detection region of the image detecting device and a logical link would have to be established for this newly added display unit. In this respect the measures of the invention are again to be viewed in the overall context of a large supermarket for instance, in which the total of all the shelves for hundreds or even thousands of product groups would have to be continuously detected by the image detecting devices installed in the shop and the masses of image data generated in the course of it would have to be transmitted to the linking device, where the computer-aided proverbial “search for the needle in the haystack” for a potentially newly added display unit would then have to be carried out i.e. by means of expensive computerised image data evaluation. The necessary network components for transmission of the image data as well as the computing power of the linking device would be enormous.
In a move away from this the computing power required for the linking device is focused or bunched on the relevant event, i.e. the appearance of a newly added display unit, which is considerably more efficient. Depending on how the invention is implemented, as will be discussed in detail further below, the volume of data as well as its processing can thus be optimised in terms of time, contents as well as geography.
The logical linking established in the course of it, in which unique identification data of the display unit are associated or paired with unique identification data of the product or product group is digitally stored in a data base.
Further particularly advantageous realizations and further developments of the invention will be found in the dependent claims as well as in the description below.
The invention discussed here is seen as being applied in particular to an ESL system. In such a system there exists a multitude of such display units, i.e. ESLs, in order to visualize product and/or price information for a presented product or product group, respectively.
Such ESLs may be mounted on shelves or the shelf rails thereof. The mounting structure, on which the display unit and the product are located, may therefore be realised as a shelf or shelf level. Mounting on a presentation table is also possible. The mounting structure may thus also be realised as a table. The ESLs may also be realised as a display label for deployment on a table. Further such a display unit may be realised as a pendant for attachment to a piece of clothing. Or the mounting structure may be arranged as a link between the electronic display and the product. Such a mounting structure may for example also be realised as a stitching thread, needle or similar. For the sake of simplifying explanation of the invention an ESL discussed hereunder with reference to the display unit will always be one which is attached to a shelf, or in concrete terms to the shelf rail thereof.
For the purpose of visualisation of information each ESL comprises a screen which is commonly realised as an electrophoretic screen, in order to save as much energy as possible in the operation of battery-powered ESLs. Other types of screen such as for example LCD screens or similar may however also be used. Alternatively a line-bound power supply may be provided for the ESLs.
In such an ESL system there also exists a multitude of access points, which form a radio network for the communication-technical supply of the ESLs. Normally a group of ESLs is assigned, respectively, to an access point. The assignment functions by choosing a suitable radio channel in the ISM band, which is used by the respective access point, and registering the respective ESL with the appropriate access point. Radio communication may be via an essentially standardised radio protocol, such as for example ZigBee® or also Bluetooth®. A proprietary radio protocol such as e.g. the protocol disclosed in WO 2015/124197 A1 may also be used.
The access points themselves are typically cable-bound to a central control device such as for example a local server, on which a software application (e.g. the shop management software mentioned in the beginning) is executed for managing and controlling the ESLs, or alternatively connected to a cloud-based software application, which offers the functionality mentioned.
The linking device addressed in connection with this invention is also realised by said local server with a separate software executed thereon or as mentioned by a cloud-based separate software solution, which is made available via one or more external servers, or it is implemented as a part of the previously mentioned shop management software. Analogously thereto a database may be provided locally or on cloud, with the aid of which the respective logical link is stored. This software provides all functions, which serve to look for, find and ultimately identify the unlinked display unit in a digitised image or also in a video, i.e. on the basis of the image data. To this end intrinsically known algorithms for image/pattern recognition are used. The same applies to the search for a product positioned correspondingly (closely) to the found unlinked display unit in said digitised image or video. These algorithms may also be supported by artificial intelligence or be entirely based thereon.
Identifying the electronic display unit/the ESL and the product in the image data may accomplished by using pattern recognition, which examines the image data for the form and/or colour of the ESLs as well as of the products. Or pattern recognition may be used which examines the image data for letters, barcodes, QR codes or similar in order to perform identification on this basis. Or image data of different points in time (e.g. prior to the occurrence of the trigger signal and after the occurrence of the trigger signal) may be used in order to perform identification on the basis of the differences in the images or videos represented by the image data. Or the blinking of an LED (LED here means “light emitting diode”) of the electronic display unit may be used in order to identify it in the image data with the aid of a unique identification code communicated by the blinking signal.
The image data analysed in the course thereof originate from the image detecting device, which may be realised as a still camera and/or video camera.
The trigger stage may be arranged to detect the presence of an unlinked electronic display unit on the mounting structure in an automatic manner and/or as a consequence of a user interaction.
If the trigger state is arranged to detect the presence of an as yet unlinked display unit on a mounting structure such as the shelf rail in an automatic manner, the trigger signal can be automatically generated and emitted directly without any further help from a staff member of the shop when attaching or positioning the display unit on the shelf rail in the vicinity of the associated or corresponding product, for which the product and/or price information has to be displayed by means of the newly added display unit. This results in an extremely quick and completely automated triggering of establishing the link.
If the trigger stage is arranged to detect the presence of an as yet unlinked electronic display unit on the mounting structure as a consequence of a user interaction, the fully automatic establishing of a logical link can be initiated or triggered in a temporally flexible and, at the same time, user-controllable manner by said user interaction of a sales assistant. As such a sales assistant has to merely perform a user interaction, for example press a button or operate a touch-sensitive element on the electronic display unit, whereupon the trigger signal is generated and emitted without any further help from the sale assistant, whereupon, as discussed, the linking process is started automatically. In this way the logical link of an already positioned electronic display unit to a new product can be triggered by the sales assistant, the product not having been placed on the shelf unit after the corresponding display unit has been positioned. The sales assistant can thus actively define or co-determine the point in time of emitting the trigger signal. The user interaction can, however, also be performed with the aid of an (external) user device of the staff member such as a personal digital assistant (e.g. realised as a tablet computer) or a mobile telephone of the staff member, wherein using this device an NFC signal (NFC here stands for “near field communication”) or a Bluetooth signal is sent to the trigger stage, which initiates/triggers the fully automatic establishment of a logic link.
If the trigger stage is arranged to detect the presence of an unlinked electronic display unit on the mounting structure in an automatic manner and as a consequence of a user interaction, the mentioned advantages of these measures can be combined or the one or other measure can be used in order to initiate or start the logical linking process.
This means that a particularly time-saving and above all user-independent method can be realised if detection is performed automatically. In this respect it has proven to be advantageous if for the purpose of automatic detection during attaching the electronic display unit on the mounting structure an electronic circuit is influenced according to one the measures listed hereunder, namely:
The embodiments b) and c) may for example be in the form of oscillating circuits, which in essence form the influenceable circuit.
All these measures may occur singly or in combination with each other and are preferably used in the implementation variants listed.
As discussed a particularly flexible method may be realised, if detection of the presence of an unlinked display unit is performed as a consequence of a user interaction. Here it has proven to be advantageous if for the purpose of detecting a user interaction a change of state of an electronic user interface is determined. The user interface may for example be touch-sensitive or even pressure-sensitive zones of the housing of the display unit or a touch-sensitive screen of the display unit. By means of an electronic circuit of the display unit a user interaction detected by means of these touch-sensitive and/or pressure-sensitive elements may be processed further and this may also subsequently be used for generating and sending the trigger signal.
As has already been addressed, according to one embodiment the electronic display unit may comprise a trigger stage and generating the trigger signal is performed by the electronic display unit itself. It is thus ensured that the electronic display unit can autonomously start, i.e. trigger the logical linking process independently of other system components. This embodiment presumes that the individual electronic display units comprise the electronic components necessary for realising the trigger stage. Such electronic display units offer the advantage that they can be positioned at any positions, i.e. can be positioned at random and are not dependent on any other system components in order to generate and send the trigger signal.
On the other hand, again as already addressed, the electronic display unit, which according to another embodiment is arranged on a shelf rail for the electronic and/or communication-technical supply of at least one electronic display unit, may comprise the trigger stage, and generating the trigger signal is performed by the electronic display unit. This has the advantage that for each mounting structure, i.e. for example the shelf rail, there exists a central instance, which starts the logical linking. The display units to be installed on the respective shelf rail may be of relatively simple construction and can therefore be manufactured in a relatively low-cost manner. The electronics required for the trigger stage is then, for each shelf rail, integrated only once in the power supply unit. Preferably the electronic display unit is arranged on a shelf rail for electronically or communication-technically supplying power to a number of electronic display units. The number of required electronic components in the system can thus be considerably reduced, which reduces the cost for both providing and maintaining the system.
Preferably the trigger stage comprises a radio signal transmitting stage, and a radio signal, in particular a Bluetooth-Low-Energy radio signal, is sent as the trigger signal by the radio signal transmitting stage. In particular due to the realisation by means of a Bluetooth-Low-Energy radio signal it is possible to perform signalling which is detectable only locally in close vicinity of the trigger stage.
Emitting the trigger signal can also be done by quite generally using a so-called beacon or radio beacon, which identifies itself by its beacon characteristic. It is therefore not, of necessity, limited to using a Bluetooth-Low-Energy radio signal.
Reception of this trigger signal is thus restricted to the field of propagation of the trigger signal limited by means of the chosen signal strength or transmission power. A receiver, the subject of which is still to be discussed further below, must therefore be positioned within a region, in which the trigger signal can be received and utilised. This results in a natural spatial delimitation around the device transmitting the radio signal, so that in order to find the unlinked display unit only the image data within this spatially limited region need to be evaluated. Naturally another radio signal can also be used, which has a larger range or which can be received only very locally.
Optionally the trigger stage may comprise a light signal emitting stage and a light signal, in particular an LED light signal, is emitted as the trigger signal by the light signal emitting stage. This light signal may be located in the visible spectrum, preferably however not in the visible spectrum, so as not to irritate the customers in the shop. Here again only a spatially limited region is under consideration, in which the emitted light signal can be received by a receiver. At the same time however, the light signal can make evaluation of the image data easier because it can be detected as such in the image data. By detecting the light signal in the image data, conclusions can be drawn as to the position of the as yet unlinked display unit or how to delimit its position in the detection region of the image detection unit.
In this context it should also be mentioned that the light signal can be used—given the computerised fully automatic evaluation of the image data—to detect for example the position of product presentation tables or shelf floors or even shelf rails by means of the linking device.
Light signalling, quite generally, thus makes identification of the horizontal levels in shelves easier.
According to a further aspect of the invention a trigger signal receiving stage is provided, with which the trigger signal is received and the receiving of the trigger signal is used as trigger for data processing.
In line with the options listed for the trigger signal, the receiving stage may be a light-sensitive electronics or a radio signal receiver.
The trigger signal receiving stage, according to one embodiment, may be arranged to check, whether the received beacon is destined for it and its associated camera, i.e. whether this is a beacon, which is admissible for the purpose of linking (“matching”) an ESL to one of the products, and in this case to instruct the camera, to begin recording and to generate and emit image data. Checking the beacon ahead as to whether this is assigned to the appropriate trigger signal receiving stage may be of advantage in particular if in the shop further beacons have to be reckoned with, which are emitted by other radio systems or radio devices for other purposes and which can also be received by the trigger signal receiving stage. Checking the beacon in advance of the start of the cameras assigned to the respective signal receiving devices is of advantage insofar as in this way the amount of image data to be further processed is restricted to the absolutely necessary extent—i.e. to the image data generated by the one single camera or very few cameras. The upstream checking of the beacon thus represents a filter in order not to admit all beacons possibly receivable in the shop as trigger for generating image data. A further natural filter results from the preferably small range of the beacon (such as e.g. a preferably used Bluetooth® Low Energy Beacon), which limits the number of trigger signal receiving stages receiving this beacon.
According to a second embodiment the trigger signal receiving stage may however also be arranged to start the camera directly upon receipt of a beacon, so that the image data are generated and output. Starting the camera without previously checking the beacon as to whether this is assigned to the receiving trigger signal receiving stage, may be of advantage then, when it is assumed that no other interfering beacon is to be expected, i.e. when for example the beacon envisaged for “matching” is emitted on a radio channel not otherwise used.
According to one embodiment the trigger signal receiving stage controls the image detecting device and the trigger for data processing causes the image detecting device to start generating the image data.
In this embodiment the start of transmitting the image data and their receipt at the linking device causes the start of the logical linking process at the linking device. In the present case the formulation “in accordance with the occurrence of the trigger signal” mentioned in the beginning means that the process of logically linking the as yet unlinked electronic display unit to the respective product is not performed until the trigger signal has triggered the generation and transmission of the image data and these image data have arrived at the linking device.
According to a further embodiment the trigger signal receiving stage activates the image detecting device and the trigger for data processing causes embedding of the meta data in the image data by the image detecting device. These meta data mark the relevant point in time or time range in the image data or in their data stream, at which the presence of the unlinked display unit was indicated by the trigger signal.
In this embodiment the occurrence of the meta data in the image data received at the linking device starts the process of logically linking at the linking device. Expressed in other words this means that the meta data are detected in the image data received at the linking device by the linking device and in consequence the process of logically linking the as yet unlinked electronic display unit to the product is started. In the present case the formulation “in accordance with the occurrence of the trigger signal” mentioned in the beginning means that the process of logically linking the as yet unlinked electronic display unit to the respective product is not performed until the trigger signal has triggered the embedding of the meta data in the image data and the linking device has detected the meta data in the received image data.
The meta data mentioned here may be in a predefined data format such as for example an unequivocally identified binary code, which is already known to the linking device and/or is embedded in a deferential manner in the image data or the data stream thereof.
In the last two mentioned embodiments the trigger signal receiving stage may be designed as a separate device which controls the image detecting device via a signal line. Preferably however, the trigger signal receiving stage is an integral constituent of the image detecting device such as for example a Bluetooth receiver. As soon as the receiving stage receives the trigger signal, generation and transmission of the image data is started.
According to a further embodiment the trigger signal receiving stage directly controls the linking device and the trigger for data processing causes a start of the process of logically linking at the linking device. According to this embodiment image data from one or a number of image detecting devices may be transmitted continuously to the linking device. The CPU-intensive process of logically linking the as yet unlinked display unit to the respective product is however not started until the trigger signal receiving stage triggers it. In this embodiment the receiving stage can also be provided as a separate module or device, which controls the linking device by means of a control signal. Preferably the trigger signal receiving stage is however realised here too as an integral part of the linking device.
In all the embodiments discussed the linking device looks for and identifies the unlinked electronic display unit in the generated image data. Subsequently it searches for and identifies in the generated image data a correspondingly positioned product for the position of the found and identified electronic display unit, which product is not as yet linked to an electronic display unit, and generates logical linking data for the purpose of logically linking the identified electronic display unit to the identified product and stores these data in a digital manner.
In order to simplify the search and the identification of the unlinked electronic display unit in the image data for the software to be used for this purpose, it may be provided that each display unit comprises its own light-emitting device. This may be a separate LED, which is preferably used in addition to an electrophoretic screen, or it may also be the background lighting of for example an LCD screen. As soon as the sending of the trigger signal is started at the display unit, a light signal is also emitted by means of the LED, which makes finding it easier in the depicted shop region detected by means of the image detecting device. The light signal may be in the form of one-time blinking, multi-times blinking or a permanent light emission or even a modulation of the intensity of the light signal.
It may however also be provided that such a light-emitting device is provided only once for each power supply device. In this configuration at least the light-emitting device can be identified in said depiction. Starting from the position of the light-emitting device, that region, in which finding the display unit is highly probable, can then be easily delimited, because for example when using one shelf rail and one power supply device per shelf rail, it is assumed that the display unit to be found can only be positioned on the left-hand or right-hand side of this power supply device along the shelf rail. The direction of the search can however be restricted still further, if the as yet unlinked display units are to be looked for only on the right-hand side or only on the left-hand side of the power supply device, namely because the power supply device can only be installed on the left edge of the shelf rail or only on the right edge of the shelf rail.
According to a further aspect of the invention it may also be provided that with the aid of the light-emitting device, in particular the LED, which is formed either on the display unit or on the power supply device, a coded light signal is emitted, with the aid of which identification of the respective searched for display unit is made easier or that even the identification code as such, i.e. for the example the code, which is also indicated as barcode on the screen of the display unit to be searched for, is communicated.
Furthermore it may be provided that the image detecting device can also focus or zoom onto a partial region of the detected total region. This may be helpful in particular if it is possible to delimit as previously discussed each image region, in which the probability is high that the unlinked display unit will be found. Preferably focusing or zooming with the image detecting device is performed digitally, i.e. without mechanical components. With the linking device the image data generated therewith can be evaluated substantially quicker and more accurately than would be case if the entire region would have to be evaluated.
In conclusion it should be mentioned quite generally that the discussed electronic devices comprise of course electronics. The electronics may consist of discrete electronics or integrated electronics or from a combination of both. Microcomputers, micro controllers, application-specific integrated circuits (ASICs), possibly in combination with analogue or digital electronic peripheral components may also be used. Radio devices normally comprise an antenna configuration as part of a transceiver module for sending and receiving of radio signals. The display units are preferably battery-driven.
These and further aspects of the invention are revealed in the figures discussed hereunder.
The invention will now be discussed once more in detail with reference to the attached figures by way of exemplary embodiments, to which the invention is however not restricted. In the different figures identical components are marked with the same reference symbols, in which schematically:
The system is installed in the premises of a supermarket, of which in the present case for the purpose of discussing the invention only a small cut-out is depicted. The drawing shows a product presentation device, which is realised as shelf 2. The shelf 2 comprises four shelf levels 4a, 4b, 4c, 4d. One shelf rail 3a, 3b, 3c, 3d is provided for each shelf level 4a to 4d. The whole supermarket comprises of course a multitude of such shelves 2, as well as further electronic components of the system 1, wherein these are not visible because the drawing only shows a cut-out.
The system 1 comprises a camera 9 with a signal receiving device 10, an access point 11 and a linking device 12 and the mentioned ESLs 5a to 5n. The ESLs 5a to 5e are attached to the front of the shelf rails 3a to 3d. For the sake of clarity and in order not overload the figures, not all ESLs 5a to 5n have been marked with reference symbols.
Each ESL 5a to 5n comprises a screen 6 and a light-emitting diode, abbreviated to LED 7, which is representative for all ESLs 5a to 5n and is indicated with its reference symbol 7 only for the lowermost ESL 5n. The screen 6 is thus realised as an electrophoretic screen and is destined, as long as ESL 5a to 5n is not linked, to display a barcode identifying the ESL 5a to 5n and as soon as the ESL 5a to 5n is linked, to display product and/or price information. LED 7, controlled by electronics (not shown in detail) of the ESL 5a to 5n, is provided to emit a light signal. The light signal here is a coded blinking, which in a temporal context of blinking permits identification of the respective ESL 5a to 5n and in a spatial context permits easier finding of the ESL in the image detected by camera 9.
Each ESL 5a to 5n also comprises a Bluetooth-Low-Energy-Beacon transmitter 16, also called a beacon transmitting stage, (see also
Each ESL 5a to 5n also comprises a radio system 17 (see
Furthermore one can see one of many detection units (framed by a broken line), which—as shown—comprises the camera 9 and a signal receiving device 10.
The linking device 12 in the present case is realised by a local computer in the supermarket, on which a shop management software is executed, which provides all functions, which are required in a supermarket in order to manage the stock of products 8a to 8e, their price information or associated product information, and also in order to subsequently generate the pairing or logical connection of the products 8a to 8e to the ESLs 5a to 5n and/or to provide this in form of a planogram for simple visualisation. The linking device 12 is thus realised as a central linking unit and connected via cable to several access points 11 as well as to several detection units, wherein the access points 11 are distributed in the shop in such a way that a radio connection is possible with all ESLs 5a to 5n, and wherein the detection units are arranged in a distributed manner, i.e. positioned and aligned in the shop in such a way that all ESLs, which are scheduled to be detected by means of the detection devices, can be detected in this way. In this embodiment each detection device is concretely installed in the ceiling of the shop in such a way that at least one shelf can be completely optically detected with it.
The access point 11 provides the radio-technical connection between the linking device 12 and the ESLs 5a to 5n, wherein product and/or price information stored in the linking device can be transmitted via this radio-technical connection to the ESLs 5a to 5n, so that each ESL 5a to 5n—as soon as it is logically linked to the respective product or the respective product group—can display individual product and/or price information. Each ESL 5a to 5e therefore must be linked or matched to a product 8a to 8e, so that the respective ESL 5a to 5n can display the relevant product and/or price information. Also, the device status of each ESL 5a to 5n can be queried via this radio-technical connection, and, as required, can be bidirectionally communicated. Normally the ESLs 5a to 5n are assigned group-wise to different access points 11, which is, however, not discussed any further herein.
The camera 9 is arranged to digitize an optical depiction of the shelf 2 and to transmit these digitised depictions in the form of image data to the linking device 12, where these are processed. These image data may for example represent stills or videos.
The signal receiving device 10 is arranged to receive radio signals, in particular the beacon of one of the ESLs 5a to 5n and in consequence thereof to trigger further actions. To this end the signal receiving device 10 comprises a Bluetooth-Low-Energy receiver.
The method mentioned in the beginning will now be discussed in detail. In
In this embodiment the ESL 5a itself comprises a detection device (not shown in detail), which detects fitting the ESL 5a into the shelf rail 4a. This may be realised for example by means of a contact on the ESL 5a, wherein the contact may for example be actuated by a pin protruding from the housing while the shelf rail is being fitted, and the actuated contact closes a circuit of the electronics of ESL 5a, which is detected by the detection device on the basis on a current flow through the contact. This may however also be triggered by an external trigger such as e.g. with the aid of a portable NFC-capable device of a staff member, which sends an NFC signal to the ESL 5a, the ESL also being NFC-capable, which is detected by the detection device.
In the present case the contact together with the electronics of the ESL 5a forms a trigger stage, which also comprises a radio signal transmitting stage for emitting the beacon as trigger signal for triggering the process of logical linking.
In consequence of detecting the fitting of the ESL 5a to the shelf rail 4a three actions are triggered/started in this embodiment by the ESL 5a fitted just now to the shelf rail.
Firstly the not yet logically connected ESL 5a starts an identification mode and displays a graphic identifying the ESL 5a on the screen 6 of the newly fitted ESL 5a. In this case the identifying graphic is the barcode, which unequivocally identifies the ESL 5a. This barcode is then used to link or match the ESL 5a to the product 8a, i.e. to establish the logical connection between the respective product 8a and the ESL 5a fitted just now into the shelf rail 3a corresponding to the products 8a and to store it in the linking device 12.
Secondly the ESL 5a just fitted into the shelf rail 3a initiates the emission of the light signal as a blinking signal by the LED 7. The frequency of blinking may be constant or variable. In this embodiment the LED 7 blinks with a variable frequency, to be exact blinks in a rhythm, which allows identification of the ESL 5a. This second action is subsequently utilised to ascertain the position of the newly fitted ESL 5a on the basis of the image data generated by means of the camera 9.
Thirdly the Bluetooth-Low-Energy-Beacon transmitter of the ESL 5a just fitted into the shelf rail 3a initiates emission of the beacon. The third action is subsequently utilised to start image detection with the camera 9, for which the beacon can be received by means of its signal receiving device 10, because it is in the receiving region. The beacon thus acts as a trigger for image detection with the camera 9, which forms the basis for an automatic matching of products 8a or 8b on the shelf level 4a to the newly added ESL 5a on this shelf level 4a. The signal receiving device 10, also called trigger signal receiving stage, receives the beacon emitted from the upper left ESL 5a and checks, whether this is a beacon, which is admissible for the purpose of matching an ESL to one of the products. The signal receiving device 10 has been programmed in advance in such a way that it can distinguish between a “correct” beacon emitted by one of the ESLs 5a to 5n and any other beacons possibly appearing in the shop. When the signal receiving device receives a beacon provided for matching, as may be assumed in this case, it activates the camera 9, so that this outputs image data to the linking device 12.
The occurrence of the “correct” beacon therefore informs the system 1, the camera 9 in detail and subsequently the linking device 12 about the presence of a new ESL, here concretely the ESL 5a.
The linking device 12 has the image data of the camera 9 sent to it almost in real time for fitting into the respective ESL 5a. The linking device 12 examines the image contents represented by the image data in this embodiment in the following manner/for the following features.
Firstly the linking device 12 examines the new image contents as to whether a newly added ESL (here ESL 5a) is present, which had not been present in older image contents, which were present prior to the occurrence of the trigger signal, i.e. as to whether the ESL 5a detected on the shelf 2 by the camera 9 has been newly added and focuses the further evaluation of the image contents thereon.
Secondly the linking device 12 checks the image contents as to whether the barcode, which identifies the newly added ESL 5a, is present and recognisable in the focus of the evaluation.
Further, the linking device 12 checks the image contents, as to whether it can be recognised that the LED 7 of the ESL 5a blinks or shines, in particular in the corresponding (known) rhythm. This check, analysis or evaluation can be performed later or even at the same time as the checks discussed earlier under “firstly” and “secondly”. Identifying the blinking or shining helps to delimit the approximate image region, in which the newly added ESL 5a is located. In the image data the area surrounding the blinking LED 7 can be easily determined both in terms of locality and over time. Local differences in brightness in the image contents permit focusing on the area surrounding the LED 7. Also the temporal change of the difference in brightness caused by the LED 7 in the image contents permits focusing on the region surrounding the LED 7. Other regions of the image contents by contrast show fewer significant differences in colour and brightness. When analysing the image contents the linking device 12, due to the blinking of the LED 7, can therefore quickly find the relevant region in the image contents, in which the newly added ESL 5a is located.
The linking device 12 uses the known position of the LED 7, in order to look for the barcode next to the image contents, with which the ESL 5a is identified. As soon as the barcode is found, it evaluates this and identifies the ESL 5a, which of course had already been registered in the linking device 12 for use in the supermarket.
Once the ESL 5a fitted just now into the shelf rail 3a has been found and identified in the image contents, the linking device 12 evaluates the image contents in that it detects the product 8a to be found next to the ESL 5a. In the present case this is a shelf 3 as seen in
The identification of the ESL 5a and the identification of product 8a are based essentially on pattern recognition. To this end the linking device 12 on the one hand compares the image contents to the digital images of the products stored on the computer. On the other end the image content is also checked for known patterns such as barcodes, QR codes, letters and numbers. Therefore text recognition can also be provided here. With regard to the respective product there may have also been made provision for this to be registered in advance at the linking device 12 for use in the supermarket. The product can also be registered on the basis of the analysis of the image contents and the information gained therefrom for use in the supermarket.
As soon as the ESL 5a and the product 8a have been identified, the linking device 12 establishes the logical link between the product 8a and the ESL 5a and stores this in a digital manner in a data base, which is hosted on the computer. This in fact implies automatic “matching” of the ESL 5a to the product 8a.
The product and/or price information is then communicated from the linking device 12 to the ESL 5a via the access point 11. For completeness' sake it remains to be mentioned at this point that the ESL 5a has of course previously been registered radio-technically in the known manner with the access point 11 and therefore has radio coverage because of access point 11. As soon as the ESL 5a has received the product and/or price information, it leaves the identification mode and displays the product and/or price information on its screen 6.
The process described here can be performed for all ESLs 5a to 5n (and products). In particular this all-encompassing process is performed when the system is installed for the first time or when the shelves are stacked with products and fitted with ESLs. On the basis of the image contents communicated by the cameras 9 the entire shop is digitally cartographed with the aid of the linking device 12, i.e. the position of the ESLs 5a to 5n and products 8a to 8e on the shelves and also on the shelf levels is digitally detected, with their positions, possible including a three-dimensional position indication, being stored in a digital data structure, which is used for the computer-supported creation of a planogram. The planogram obtained in this way is a visualisation of then placement of the articles on the shelves/shelf levels of the shop. In the course of operating the business there may be changes in the positions for products 8a to 8e and assigned ESLs 5a to 5n, which is however automatically detected as previously discussed and incorporated in the planogram practically in real time. The planogram is therefore dynamically updated.
In contract to
Each shelf rail control unit 13a-13d incorporates the access points 11 for radio-technical communication. Each shelf rail control unit 13a-13d supplies the ESLs 5a-5n communication-technically connected thereto with energy directly at the respective shelf rail 3a to 3d. The shelf rail control unit 13a-13d itself may be supplied with energy via cable or via a rechargeable accumulator or a battery or even via radio, such as e.g. by means of “power over Wi-Fi”.
The shelf rail control unit 13a-13d therefore represents an intermediate link in the communication of the ESLs 5a-5n with the linking device 12. Communication of the ESLs 5a-5n with the linking device 12 is therefore as before via the access point 11, but with the interposition of the respective shelf rail control unit 13a to 13d.
In contrast to the embodiment in
This embodiment has the advantages that the ESLs 5a to 5n mounted on the shelf rail 3a to 3d can be realised at low cost and substantially easier than is the case with the embodiment in
If now, for example, the upper left ESL 5a is newly fitted into the shelf rail 3a, the shelf rail control unit 13a automatically detects the appearance of the new ESL 5a. This can for example be recognised by means of the change in energy consumption/the electrical power drawn. Also, the ESL 5a can communicate a signal to the shelf rail control unit 13a, by means of which the shelf rail control unit 13a recognises that a new ESL 5a is present. Automatic signalling can also be provided via the line during fitting the ESL 5a to the shelf rail.
The newly added ESL 5a is in identification mode as mentioned and the identifying barcode is unequivocally displayed on its screen 6. Alternatively the shelf rail control unit 13a can send a signal to the ESL 5a, wherein the signal places the ESL 5a into identification mode.
Once the shelf rail control unit 13a has detected the fitting of the newly added ESLs 5a, it now emits the beacon and controls its LED 7 such that the LED 7 emits a blinking signal.
The signal receiving device 10 located in the receiving region of the beacon receives the beacon emitted by the uppermost shelf rail control unit 13a and in consequence thereof starts image detection with the aid of its camera 9. The camera 9 detects the shelf 2 also in this embodiment, generates the image data and transmits these to the linking device 12 for further processing.
The linking device 12, by means of the blinking signal of the LED 7, initially identifies in the image contents the shelf rail 4a, in which the newly added ESL 5a is expected to be found, wherein here as well the differences compared to older image contents are analysed as required in order to delimit the position of the newly added ESL 5a and to look there for the barcode of the ESL 5a. During the search for the newly added ESL 5a it is helpful that the linking device 12 knows that the ESLs 5a to 5n (extending essentially horizontally) are always positioned to the left of the respective shelf rail control unit 13a to 13d. When the barcode is found the ESL 5a is identified therewith and as before in connection with the first embodiment the associated product 8a is identified and the ESL 5a is logically linked to the product 8a, i.e. matching is performed.
In contrast to the embodiment shown in
In this embodiment the camera 9 and its signal receiving device 10 are integrated in the shelf separating device 14a and 14b. The cameras 9 are attached to the front wall of the shelf separating devices 14a and 14b and detect with their detection region in this configuration discussed here the opposite shelf 2 across the shelf aisle. In other words each of the cameras 9 detects the opposite shelf 2.
Each shelf separating device 14a and 14b also comprises a radio device 17 configured analogously to the ESLs 5a to 5n, in order to be able to communicate by means of the access point 11 with the linking device 12, in order to be able to transmit the image data of the camera 9.
The ESLs 5a to 5n are configured as in the embodiment in
The beacon emitted by the ESLs 5a to 5n triggers as discussed image detection of the cameras 9 and the linking device 12 identifies the newly added ESL as discussed and matches it with the product located at the respective position.
Combinations of the discussed embodiments are also possible. So for example, in the embodiment discussed in connection with
In a first step I the beacon is emitted in consequence of fitting one of the ESLs 5a into the shelf rail 3a, wherein, depending on the embodiment this is initiated either by the ESL 5a to 5n itself or by the shelf rail control unit, 13a to 13d.
In a second step II, in consequence of receiving the beacon at the signal receiving device 10:
In a third step III, possibly in accordance with the separate timestamp data or the predefined timestamp image element, the ESL 5a emitting the beacon and the product 8a positioned correspondingly thereto, i.e. product 8a belonging to this ESL 5a is identified on the basis of an evaluation of the received image data by the linking device 12, and the ESL 5a is logically linked to this product 8a.
The method thus ensures automatic matching of a newly added ESL 5a to 5n to a product 8a to 8n positioned correspondingly thereto in accordance with the occurrence of the trigger signal, i.e. the beacon.
Subsequently the product and/or price information, which concerns the product 8a, is provided by the linking device 12 for the respective ESL 5a and radioed via the access point 11. In the embodiments according to
As soon as this communication has finished, there can be a wait in the system for a digital confirmation message of the ESL 5a, in which the ESL 5a confirms that it has converted the received data and updated the content of its screen. This confirmation message is also communicated to the linking device 12 according to the respective embodiment either directly via the access point 11 or indirectly with the interposition of the shelf rail control unit 13a. If this confirmation message is, however, not received, taking into account a permitted timespan, up to the expiry of which the message could still arrive, a new attempt of communicating the product and/or price information to the ESL 5a can be made in the system or an alarm can be triggered, which prompts a staff member of the supermarket to check, whether the affected ESL could possibly be damaged.
With the embodiments described here it is, however, possible to also use the installed cameras 9 and the image data provided thereby to perform a check on the update of screen 6 of the respective ESL 5a to 5n at the linking device 12 on the one hand during initial supply of price and/or product information and also on the other hand during ongoing operation after each update of the respective ESL 5a to 5n.
Furthermore it can also be checked at the linking device 12 by means of the image data provided by the cameras 9, whether during ongoing operation the ESLs 5a to 5e are still positioned in the correct place corresponding to the positions of products 8a to 8e, with which they were logically linked and for which possibly an alarm was triggered in case of known deviations outside a tolerance range.
In summary the system provides the following functionalities, namely:
Generally it should be mentioned at this point that the camera 9 and its signal receiving device 10 can also be designed such that they can be mounted directly on a shelf rail 3a to 3d, i.e. that they can be attached there analogously to the ESLs 5a to 5n. Therefore there is no need for mounting them separately outside the shelves 2, and a camera 9 attached thus on a shelf 2 can detect the opposite shelf 2 across the shelf aisle 2 from the shelf 2 on which it is mounted.
In conclusion it should be pointed out once more that the figures described in detail above are merely exemplary embodiments, which may be modified by the expert in a wide variety of ways without leaving the scope of the invention. For completeness' sake it is also pointed out that the use of the indefinite article “a” does exclude that the respective features may also be present multiple times.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2021/078505 | 10/14/2021 | WO |
