Patent Application
20080077510
The invention is herein described, by way of example only, with reference to the accompanying drawings, wherein:
The present invention is a system that utilizes fixed video cameras for providing security and shelf monitoring functions.
The principles and operation of a system that utilizes fixed video cameras for providing security and shelf monitoring functions according to the present invention may be better understood with reference to the drawings and the accompanying description.
By way of introduction, the present invention relates to a system and a method for monitoring both the security and the current product distribution in a product storage area using the same video camera network.
Although the accompanying figures relate to shelves deployed in a retail area of the store, it should be noted that the phrase “a product storage area” is used herein to refer to substantially any area in which items are placed. Such placement may be for display in the retail section of a store, for example. Alternatively, such placement may be in a warehouse or storage area, such as the “back room” of a retail store, in which the items are placed until they are brought out to the retail section of the store for display.
The system of the present invention utilizes a network of video cameras to view a product storage area. The video image data acquired by the video cameras is directed to two different modules in the processing system, a shelf monitoring module and a security module. The video image data, therefore, is analyzed with regard to both security issues and product distribution. It should be noted that the term “video image data” is used herein to refer to substantially any output produced by the video cameras. Further, the video cameras may acquire the video image at substantially any frame rate known in the art, including rates that are considered still images. It will be appreciated that the video image data may be processed as, but not limited to, moving images and as still images. It will be understood that the sampling rate of the security module may be different from the shelf monitoring module and therefore, the processing system may send different frames to each module.
It will also be understood that not all of the video image data acquired is required for analysis by the security module and likewise, not all of the video image data acquired is required for analysis by the shelf monitoring module Therefore, the processing system may be configured so as to direct only the security relevant video image data, referred to herein as “security relevant video image data”, to the security module and to direct only the shelf monitoring relevant video image data, referred to herein as “shelf monitoring video image data”, to the shelf monitoring module. In a simplest embodiment, such relevance may be determined for the security module based on the difference between the frame rate of the video camera and refresh rate of the video display monitor. Relevance may be determined for the shelf monitoring module based on analyzing one frame, or a series of a predetermined number of frames, at a predetermined time interval such as, for example, every 30 minutes, hourly, or every two hours It should be noted that at substantially any given time the video image data being analyzed by the shelf monitoring module may be the same or different from the video image data being analyzed by the security module. Therefore, the phrases “security video data” and “shelf monitoring video data” as used herein may, or may not refer to the same data being analyzed at substantially any given time.
In order to compensate for, and eliminate if necessary, objects such as shoppers in the store moving through the field of view of a particular camera, multiple frames may be processed together so that a passing shopper is not interpreted as an empty shelf This may be accomplished by comparing pixels of successive frames and rejecting any that are not constants Alternatively, if two or more cameras provide a view of the same shelve from different angles, the video image data from the two cameras may be compared and any non-corresponding data may be rejected.
It should be noted that the video cameras of the present invention may be configured to pan a predefined portion of the product storage area. Alternately, the video cameras of the present invention may be configured to provide a static view of a predefined portion of the product storage area.
The analysis of the video image data by the security module may be in the form of displaying the video image data as a video image displayed on a video monitor for surveillance viewing by security personnel. Alternately or additionally, the video image data may be analyzed by a security related computer program containing a security algorithm as is known in the art so as to further assist security personnel.
The analysis of the video image data by the shelf monitoring nodule is done in order to determine the product re-stocking needs of the products arranged on the shelves viewed by the video cameras. Therefore, at least portions of the same video image data is analyzed both for security purposes and product re-stocking needs.
It will be appreciated that the processing system may include separate processing units for the shelf monitoring module and the security module Alternatively, the shelf monitoring module and the security module may be included on a single processing unit, Output resulting from the security and re-stocking, analysis may be displayed on a single display monitor or on separate dedicated display monitors. If the output is to be displayed on a single monitor, it may be displayed using a split screen arrangement or as one window superimposed over another window. For example, the video image may be continuously displayed “full screen” on the monitor for security purposes and a pop-up window may be activated to display the re-stocking analysis results. Such a pop-up window may be activated on demand or automatically as per predetermined criteria.
Alternatively, the output from the shelf monitoring module may be in the form of, but need not be limited to, an automated audio output, an SMS message, e-mail or substantially any other type of message that is sent directly or via any available local or wide area network. Preferably, the re-stocking analysis results are sent to the storage location from which the shelves are to be replenished or to the individual responsible for such replenishment.
As will be discussed in greater detail below, a product distribution and re-stocking program included in the shelf monitoring module may be configured to analyze, by non-limiting example, the distribution of products adjacent to an outer edge of the shelf, throughout the full depth of the shelf, or solely to identify empty shelf space Such product distribution and re-stocking program may include any one or combination of suitable algorithms known in the art such as, but not limited to, shape recognition algorithms.
The program of the shelf monitoring module may also include a learning mode for introducing a new product The learning mode may include passing each new product in front of a camera for the shelf monitoring module to learn the visual signature (pattern) of the product.
The shelf monitoring module may also include a learning mode in which the system learns the characteristics of the various shelves in view. Such characteristics may include, but are not limited to, the physical dimensions and visible depth of the various shelves, and/or identifying overlap of camera fields of view.
In a particularly simple implementation, when the viewed shelves are full according to a predetermined layout, a “snapshot” for each camera view may be acquired. The snapshot is then used as a reference template which the shelf monitoring module can then test correlation to that view over ongoing processing. That is to say, tile shelf monitoring module can compare the current view, or video image data, to the video image data of the reference template. When the predetermined layout of shelf is to be changed, the store staff can then arrange the shelves according to the new layout and then acquire a “reset snapshot”.
Such an implementation is particularly useful when the system is configured to provide notification of the disarrangement of shelves, such as products not arranged according to the programmed reference template, even when the shelf space is fully covered.
The processing system may be configured to assess video coverage of the shelving, and if so indicated, additional video cameras may be added in order to enhance video coverage of the shelving. This is particularly important when retrofitting an existing security camera network to include the shelf monitoring module of the present invention.
Since most, if not all, retail stores have some kind of security surveillance system that includes video cameras, it will be readily appreciated that the system of the present invention is of value for identifying incompletely stocked or badly arranged shelves with bare space alone the front edge, even if coverage of the depth of shelf is partial, thereby providing valuable added functionality when investment in a dedicated camera-based system would be prohibitive.
Referring now to the drawings,
As illustrated by the solid-line arrows, camera 50 has a view of shelves 22a, 22b and 22c. In this illustration, the analyzing of the product distribution on the shelf so as to determine if the distribution is indicative of a need to re-stock at least a portion of the shelf may be based on the distribution of the products adjacent to the outer edge of the shelf The broken-line arrow illustrates the view of isle 30 provided by camera 50, Camera 52 has a view of shelves 20d, 20e, 20f, 24a, 24b and 24c, and of both isles 30 and 32. Camera 54 has a view of shelves 20d, 20e, 22d, 22e, and 22f, and of isle 32.
The view of the shelves provided by cameras 50, 52 and 54 is limited to the front edge of the lower shelves and to a partial depth of the upper shelves. An assessment of video coverage of the shelving provided by cameras 50, 52 and 54 may indicate the need for an additional video camera in order to enhance video coverage of the shelving.
As illustrated in
As illustrated in
It will be appreciated that the above descriptions are intended only to serve as examples and that many other embodiments are possible within the spirit and the scope of the present invention.
