The present invention relates to management of a product.
In stores, such as convenience stores and supermarkets, products are displayed on product shelves. An information processing technology has been developed to increase efficiency of an operation of displaying the products. For example, Patent Literature 1 discloses a technology for automatically determining whether or not a relationship between a product placed on a product shelf rack and a shelf label attached to the product shelf rack is correct.
There is a case where an operation is performed of prescribing a method for displaying each product on a product shelf rack (a display position, a number of faces, or the like) in advance, and displaying the product according to the prescription. However, even though the operation is performed, there is a case where the product is not actually displayed according to the prescription. Here, a work is performed in which a person or the like who is in charge of a manufacturer visually checks a display situation of the product and checks whether or not the product is correctly displayed. However, in a case where the check is visually performed, there is a problem in that a work load of a worker is large.
The present invention has been made in view of the above problems, and one object of the present invention is to provide a technology which is capable of easily grasping correctness of display of the product.
An information processing apparatus according to the present invention includes: 1) an image analysis unit that performs image analysis on a shelf rack image in which a product shelf rack is captured, and generates actual display information representing a display situation of a product on the product shelf rack, and 2) a comparison unit that compares the actual display information with reference display information representing a reference for display of the product on the product shelf rack, and generates comparison information representing a result of the comparison.
An information processing system according to the present invention includes an information processing apparatus and a mobile terminal.
The mobile terminal transmits a shelf rack image, in which a product shelf rack is captured, to the information processing apparatus.
The information processing apparatus includes: 1) an image analysis unit that receives the transmitted shelf rack image, performs image analysis on the shelf rack image, and generates actual display information representing a display situation of the product on the product shelf rack, 2) a comparison unit that compares the actual display information with reference display information representing a reference for display of the product on the product shelf rack, and generates comparison information representing a result of the comparison, and 3) a generation unit that generates a result image representing a product which does not match the reference indicated in the reference display information and a product which matches the reference indicated in the reference display information in a distinguishable aspect using the result of the comparison, and transmits the generated result image to a mobile terminal.
The mobile terminal receives the result image and displays the result image on a display apparatus.
A control method of the present invention is executed by a computer. The control method includes: 1) performing image analysis on a shelf rack image in which a product shelf rack is captured, and generating actual display information representing a display situation of a product on the product shelf rack; and 2) comparing the actual display information with reference display information representing a reference for display of the product on the product shelf rack, and generating comparison information representing a result of the comparison.
A program of the present invention causes a computer to execute respective steps included in the control method of the present invention.
According to the present invention, there is provided a technology which is capable of easily grasping correctness of display of a product.
The above-described object and other objects, features, and advantages will be further clarified by preferred example embodiments described below and the accompanying drawings attached thereto.
Hereinafter, example embodiments of the present invention will be described with reference to the accompanying drawings. The same reference numerals are given to the same components throughout the drawings, and the description thereof will not be repeated. Further, unless being particularly specified, in each block diagram, each block represents not a configuration in a hardware unit but a configuration in a functional unit.
<Overview>
The information processing apparatus 2000 acquires a shelf rack image in which a product shelf rack, on which products are displayed, is captured. In
The information processing apparatus 2000 acquires reference display information representing a reference for display of the product on the product shelf rack 20. For example, the reference display information indicates information relevant to a product to be displayed on the product shelf rack 20, a position of each product on the product shelf rack 20, and the like.
The information processing apparatus 2000 compares an actual display situation represented by the shelf rack image 12 with the reference display information, and generates information representing a result of the comparison. For example, in
In order to realize the above-described processing, the information processing apparatus 2000 performs image analysis on the shelf rack image 12, and generates information (hereinafter, actual display information) relevant to a display situation of the products on the product shelf rack 20. The actual display information indicates, for example, names of the products displayed on the product shelf rack 20 and positions of the respective products.
The information processing apparatus 2000 compares the actual display information generated by performing image analysis on the shelf rack image 12 with the acquired reference display information, and generates comparison information representing the result.
<Effects>
According to the information processing apparatus 2000 of the present example embodiment, the actual display information generated by performing the image analysis on the shelf rack image 12 is compared with the reference display information, and thus the comparison information representing a result of the comparison is generated. The actual display information is compared with the reference display information as above, and thus it is possible to grasp whether or not a display situation of the products on the product shelf rack 20 matches the reference. Therefore, it is possible to easily grasp whether or not the display situation of the products matches the reference, compared to a case where a person visually checks whether or not the display situation of the products matches the reference.
In addition, in the case where the person visually checks the display situation, there is a problem in that a human error, such as a mistake in vision, occurs. According to the information processing apparatus 2000, it is possible to prevent occurrence of the human error.
<Example of Functional Configuration>
<Example of Hardware Configuration of Information Processing Apparatus 2000>
Each functional component of the information processing apparatus 2000 may be realized by hardware (for example: a hard-wired electronic circuit) that implements each functional component, or a combination of the hardware and software (for example: a combination of the electronic circuit and a program which controls the electronic circuit). Hereinafter, a case where each functional component of the information processing apparatus 2000 is realized by the combination of the hardware and the software will be further described.
A computer 1000 is a variety of computers. For example, the computer 1000 is a personal computer (PC), a server machine, a tablet terminal, a smartphone, or the like. The computer 1000 may be a dedicated computer designed for realizing the information processing apparatus 2000 or a general-purpose computer.
The storage device 1080 stores program modules that implement respective functions of the information processing apparatus 2000. The processor 1040 realizes the respective functions corresponding to the program modules by executing the program modules.
<Camera 10>
The camera 10 is an arbitrary imaging device that is capable of performing imaging and generating an image representing a result of the imaging. For example, the camera 10 is a camera provided in a smartphone, a tablet terminal, or the like.
<Example of Usage Pattern of Information Processing Apparatus 2000>
For example, a user of the information processing apparatus 2000 uses the information processing apparatus 2000 via a mobile terminal. The user of the information processing apparatus 2000 is a person, such as a rounder, who performs an operation of checking the display situation of the products.
The mobile terminal 3000 is provided with the camera 10. In addition, an application 100 for using the information processing apparatus 2000 is installed in the mobile terminal 3000 in advance. The application 100 controls the camera 10 to perform imaging. Further, the application 100 transmits a request including the shelf rack image 12 generated by the camera 10 to the information processing apparatus 2000. The information processing apparatus 2000 receives the request from the application.
The information processing apparatus 2000 performs the image analysis on the shelf rack image 12 indicated in the request to generate the actual display information, and compares the actual display information with the reference display information to generate the comparison information. Then, the information processing apparatus 2000 transmits the generated comparison information to the mobile terminal 3000.
It should be noted that, the comparison information is not necessarily transmitted to the mobile terminal 3000. A method for treating the generated comparison information will be described later. Further, the information processing apparatus 2000 is not necessarily used via the mobile terminal 3000.
<Process Flow>
It should be noted that, the flow of the process executed by the information processing apparatus 2000 is not limited to the above example. For example, the acquisition of the reference display information may be performed before the actual display information is generated.
Timing at which the information processing apparatus 2000 executes a series of processes illustrated in
<Method for Acquiring Shelf Rack Image 12: S102>
A method for acquiring the shelf rack image 12 by the image analysis unit 2020 is arbitrary. For example, the image analysis unit 2020 acquires the shelf rack image 12 included in the request transmitted by the above-described mobile terminal 3000. In addition, for example, in response to the reception of the operation of specifying the shelf rack image 12 by the user, the image analysis unit 2020 may access the storage apparatus, in which the specified shelf rack image 12 is stored, and may acquire the shelf rack image 12.
Here, one product shelf rack 20 may be represented by a plurality of shelf rack images 12. For example, in a case where the product shelf rack 20 is long horizontally, it is possible to image whole parts of the product shelf rack 20 by imaging the product shelf rack 20 a plurality of times while shifting an imaging range of the camera 10 in a horizontal direction.
At this time, in order to ensure that all products displayed on the product shelf rack 20 are imaged, for two shelf rack images 12 including regions adjacent to each other on the product shelf rack 20, it is preferable to image the product shelf rack 20 such that a product on a boundary (ends of the shelf rack images 12) is included in both the shelf rack images 12. For example, it is possible for the information processing apparatus 2000 to grasp the situation of one product shelf rack 20 from the plurality of shelf rack images 12 by treating the plurality of shelf rack images 12 as being combined at parts including common products. This corresponds to, for example, a case where a plurality of pictures are taken such that two pictures partially overlap each other to obtain a panoramic picture.
<Method for Identifying Product Shelf Rack 20 Included in Shelf Rack Image 12>
The information processing apparatus 2000 needs to identify which product shelf rack installed in a store or the like is the product shelf rack 20 included in the acquired shelf rack image 12. For example, when the shelf rack image 12 is acquired, the information processing apparatus 2000 acquires an identifier that identify the product shelf rack 20 included in the shelf rack image 12.
For example, as described above, it is assumed that a request including the shelf rack image 12 is transmitted from the application 100, which operates on the mobile terminal 3000, to the information processing apparatus 2000. In this case, the application 100 includes an identifier of the product shelf rack 20 included in the shelf rack image 12 in the request. For example, the application 100 receives an input for specifying the identifier of the product shelf rack 20 from the user of the mobile terminal 3000, and includes the input identifier in the request.
A method for identifying the product shelf rack 20 included in the shelf rack image 12 is not limited to a method for acquiring the identifier of the product shelf rack 20 together with the shelf rack image 12. For example, the identifier of the product shelf rack 20 is caused to be imaged by the camera 10. Specifically, a mark representing the identifier of the product shelf rack 20 is attached to the product shelf rack 20 in advance. In this manner, the mark representing the identifier of the product shelf rack 20 is included in the shelf rack image 12 generated as a result of imaging the product shelf rack 20 by the camera 10. It is possible for the information processing apparatus 2000 to identify the product shelf rack 20 included in the shelf rack image 12 by performing the image analysis on the mark. For example, it is possible to use a character string representing the identifier of the product shelf rack 20, a QR code (registered trademark), or the like, as the mark.
<Generation of Actual Display Information: S104>
The image analysis unit 2020 generates the actual display information by performing the image analysis on the shelf rack image 12 (S104). Hereinafter, information included in the actual display information will be described.
<<Identification Information of Product>>
The actual display information includes identification information of each product displayed on the product shelf rack 20. For example, the identification information of the product is a product name, an identifier, such as a Japanese Article Number (JAN) code, or the like. Hereinafter, unless otherwise specified, description will be given on the assumption that the product name is used as the identification information of the product. The same applies to the reference display information which will be described later.
For example, the image analysis unit 2020 extracts an image region representing each product from the shelf rack image 12. The image analysis unit 2020 acquires information relevant to the product represented by the image region from a product database by comparing the extracted image region with the image feature of the product registered in the product database. In the product database, it is assumed that, for each product, any of pieces of information, such as at least the product name, the identifier (such as the Japanese Article Number (JAN) code), a name (for example, a brand name) of a group to which the product belongs, a product category to which the product belongs, and a content of the product (for example, 350 ml), a size of the product, a standard price of the product, and manufacturer information is registered. In addition, a manufacturer of the product is an arbitrary manufacturer (for example, a manufacturer or a sales manufacturer) related to the product.
It is possible for the image analysis unit 2020 to identify the product name of each product included in the shelf rack image 12 based on the information acquired from the product database as described above.
<<Group Name>>
In addition, for example, the actual display information indicates a name of a group to which the product belongs. For example, similar to the above-described product name, it is possible to acquire the group, to which the product belongs, from the product database. It is possible to use any information, such as the identifier of the manufacturer, which is capable of identifying the manufacturer, instead of the name of the group. The same applies to the reference display information which will be described later.
<<Manufacturer Name>>
In addition, for example, the actual display information indicates a manufacturer of the product. For example, similar to the above-described product name, it is possible to acquire the manufacturer of the product from the product database. It is possible to use any information, such as an identifier of the manufacturer, which is capable of identifying the manufacturer, instead of a manufacturer name. The same applies to the reference display information which will be described later.
<<Position>>
In addition, for example, the actual display information indicates a position of the product on the product shelf rack 20. The position of the product is indicated by, for example, a combination of a position in a stage direction (for example, the product is positioned on what stage from a top stage) and a position in a column direction (for example, the product is positioned in what column from a leftmost column). Here, it is possible to use an existing technology as a technology for determining the position of each product from an image including the product shelf rack.
In a case where the same type of product (product having the same product name) is disposed at a plurality of positions, the actual display information indicates the plurality of positions for one product.
<<Number of Faces>>
In addition, for example, the actual display information indicates the number of faces of the product. It should be noted that, there may be a case where the same product is separately displayed on the product shelf rack 20.
In the case where the same product is disposed in places separated from each other in this way, for example, the actual display information individually indicates the number of faces for the respective places separated from each other. For example, in the case of
In addition, for example, in the actual display information, a value acquired by summing the faces in the plurality of places separated from each other may be used as the number of faces of the products. For example, in the case of
<<Example of Configuration of Actual Display Information>>
Here, information to be included in the actual display information, that is, information to be acquired by performing the image analysis on the shelf rack image 12 may be set in the image analysis unit 2020 in advance, or may be dynamically determined. In the latter case, for example, the image analysis unit 2020 determines the information to be acquired from the shelf rack image 12 based on the acquired reference display information. For example, in a case where the reference display information indicates a product to be placed on the shelf rack and a position where each product should be placed, the information to be acquired from the shelf rack image 12 by the image analysis unit 2020 is the product name and the position of the product.
<Reference Display Information>
As described above, the reference display information indicates the reference for the display of the product on the product shelf rack 20. Hereinafter, information indicated in the reference display information will be described.
<<Identification Information of Product>>
For example, the reference display information indicates identification information (including the product name, the JAN code, or the like) of each product to be displayed on the product shelf rack 20. In the description below, unless otherwise specified, it is assumed that the identification information of the product indicated by the reference display information is the product name.
<<Position>>
In addition, for example, the reference display information indicates a position where each product should be placed on the product shelf rack 20. The position of the product indicated by the reference display information is represented by, for example, the combination of the position in the stage direction and the position in the column direction. In this case, the reference display information indicates that both the position of the product in the stage direction and the position in the column direction should be the positions indicated by the reference display information.
The position of the product indicated by the reference display information may be a range of the position in the column direction or a vertical direction. For example, the reference display information indicates a range such as “second to fourth stages” or “third to fifth columns”. In addition, for example, the reference display information may indicate only a position in the stage direction for a certain product. In this case, it is possible to display the product in any column in the stage indicated in the reference display information. Similarly, the reference display information may indicate only the position in the column direction for a certain product.
<<Number of Faces>>
In addition, for example, the reference display information indicates the number of faces of the product. For example, a case where the reference display information indicates 5 as the number of faces of the product A indicates that the product A should be displayed such that the number of faces is 5.
<<Parallel Request>>
In addition, for example, the reference display information may indicate that, for a certain product, the product should be collectively disposed (parallel request). In other words, it means that the product, for which the parallel request is indicated in the reference display information, should not be separately disposed in a plurality of areas, as illustrated in
Here, a unit, in which the parallel request is specified, is not limited to the product. For example, the parallel request may be specified in units of a product group. In this case, for example, the reference display information indicates the parallel request for products of a determined brand. Each product, which belongs to the brand for which the parallel request is indicated in the reference display information, is required to be disposed at a collective position on the product shelf rack 20.
In addition, for example, the parallel request may be specified in units of a manufacturer. In this case, for example, the reference display information indicates the parallel request for a product of a determined manufacturer. Each product of the manufacturer, for which the parallel request is indicated in the reference display information, is required to be disposed at the collective position on the product shelf rack 20.
In addition, for example, the parallel request may be specified by the content of the product. In this case, for example, the reference display information indicates the parallel request for a product having a determined content. Each product, for which the parallel request is indicated in the reference display information and which has a determined content, is required to be disposed at the collective position on the product shelf rack 20. More specifically, for example, in a case where the determined content indicates 500 ml, a product having a content of 500 ml, for example, a canned beer of 500 ml sold by the manufacturer is disposed at a collective position on the product shelf rack 20.
<<Arrangement Request>>
In addition, for example, the reference display information may indicate a request (hereinafter, an arrangement request) relevant to a disposition relationship among a plurality of products. For example, the arrangement request is a request to “dispose the product A and a product B to be adjacent to each other”. For example, it is assumed that the reference display information indicates the arrangement request without indicating the positions for the product A and the product B. In this case, the product A and the product B may be disposed at arbitrary positions on the product shelf rack 20 as long as they are adjacent to each other.
In contrast, the arrangement request may indicate that multiple products should be separately placed. For example, it is conceivable to define an arrangement request such as “the product A and the product B are disposed in different stages”.
Here, a unit, in which the arrangement request is specified, is not limited to the product. For example, the arrangement request may be specified in units of a product group. For example, there is a request to “dispose a group D and a group E to be adjacent to each other”. In this case, it is necessary that, in the product shelf rack 20, a region where the products belonging to the group D are disposed and a region where the products belonging to the group E are disposed are adjacent to each other. It should be noted that, similar to the product arrangement request, the group arrangement request may indicate that a plurality of groups should be separately disposed.
In addition, for example, the arrangement request may be specified in units of a manufacturer. In this case, for example, there is a request to “dispose a product of a manufacturer X and a product of a manufacturer Y to be adjacent to each other”. In this case, it is necessary that, in the product shelf rack 20, a region where the product of the manufacturer X and a region where the product of the manufacturer Y are disposed to be adjacent to each other. It should be noted that, similar to the product arrangement request, the manufacturer arrangement request may indicate that the products of a plurality of manufacturers should be disposed to be separated from each other.
<<Applicable Period>>
In addition, for example, the reference display information may indicate, for each product, a period during which the reference display information is applied to the product. For example, a case where the reference display information indicates an applicable period of “10/01/2017 to 10/31/2017” for the product A means that it is necessary to observe information (for example, the position) for the product A indicated by the reference display information in the applicable period. The applicable period may be defined for a group of the products or the entire reference display information, instead of each product.
<<Example of Configuration of Reference Display Information>>
Data is indicated in any one of the product 502, the group 504, and the manufacturer 506. A record, in which the product name is indicated in the product 502, indicates a reference relevant to the product having the name. A record, in which the group name is indicated in the group 504, indicates a reference relevant to the group having the name. A record, in which the manufacturer name is indicated in the manufacturer 506, indicates a reference relevant to the manufacturer having the name. A product to be disposed on the product shelf rack 20 is a product whose product name is indicated in the product 502. It should be noted that, the table 500 described with reference to
The reference display information may be information relevant to all products placed on the product shelf rack 20 or information relevant to some products. In the latter case, for example, the reference display information indicates a reference for display of a product of the determined manufacturer. In this case, the reference display information for one product shelf rack 20 is prepared for each manufacturer. Therefore, the reference display information in
In a case where the reference display information is prepared for each manufacturer, the information processing apparatus 2000 may compare only the reference display information of the determined manufacturer with the actual display information. In this case, for example, the information processing apparatus 2000 acquires information (for example, a manufacturer name) for determining the reference display information which is a target of the comparison. This information is input by the user of the information processing apparatus 2000. For example, the user performs an input operation of specifying the manufacturer name with respect to the above-described mobile terminal 3000. The application 100 transmits the manufacturer name, which is input by the user, to be included in the above-described request. The information processing apparatus 2000 receives the request, and thus it is possible to grasp the manufacturer name specified by the user. In addition, for example, the user may input a manufacturer name by operating an input device connected to the information processing apparatus 2000.
<Method for Acquiring Reference Display Information: S106>
There are various methods for acquiring the reference display information by the information processing apparatus 2000. For example, the reference display information is stored in advance in the storage apparatus which is accessible from the information processing apparatus 2000. In this case, the information processing apparatus 2000 acquires the reference display information by accessing the storage apparatus. At this time, the information processing apparatus 2000 acquires the reference display information relevant to the product shelf rack 20 by using the identifier of the product shelf rack 20 which is a target of processing. In a case where the reference display information of the determined manufacturer is acquired, the information processing apparatus 2000 acquires the reference display information of the determined manufacturer by using the manufacturer name or the like which is acquired using the above-described method.
In addition, for example, the reference display information may be stored in the storage apparatus included in the management server. In this case, for example, the information processing apparatus 2000 acquires the reference display information of the product shelf rack which is determined by the identifier, by transmitting a request including the identifier of the product shelf rack 20 to the management server. In a case where the reference display information of the determined manufacturer is acquired, the identifier of the manufacturer (for example, the manufacturer name) is also included in this request.
<Comparison of Actual Display Information with Reference Display Information: S108>
The comparison unit 2040 compares the actual display information with the reference display information, and generates comparison information representing a result of the comparison (S108). For this reason, the comparison unit 2040 determines whether or not each reference (each reference for the product, the group of products, or the manufacturer) indicated by the reference display information is satisfied in the actual display information.
For example, the comparison unit 2040 determines whether or not each product, which is indicated in the reference display information, is indicated in the actual display information by comparing the product name (identification information of the product) indicated in the actual display information with the product name (identification information of the product) indicated in the reference display information.
In addition, for example, in a case where the reference display information indicates a position of a certain product, the comparison unit 2040 determines whether or not the position of the product in the actual display information matches the position indicated in the reference display information.
In addition, for example, in a case where the reference display information indicates the number of faces of a certain product, the comparison unit 2040 determines whether or not the number of faces of the product indicated by the actual display information matches the number of faces indicated in the reference display information. Here, the number of faces of the product indicated by the actual display information may be larger than the number of faces of the product indicated by the reference display information. For example, a situation may arise in which, for a product that should have three faces, the number of faces is five in an actual display situation. In this way, in a case where the number of faces of the product in the actual display situation exceeds the number of faces indicated by the reference display information, the product may be determined that the number of faces matches the reference or does not match the reference. Information for determining a result of the determination on either matching or non-matching is included in, for example, the reference display information.
In a case where the reference display information indicates the parallel request or the arrangement request, the comparison unit 2040 determines whether or not the parallel request or the arrangement request is satisfied on the basis of the position of each product indicated in the actual display information. For example, it is assumed that the reference display information indicates the parallel request representing that products, which belong to a certain product group, should be collectively placed. In this case, the comparison unit 2040 determines whether or not the products of the product group are collectively placed on the basis of the positions of the respective products, which belong to the product group, in the actual display information. Specifically, in a case where the positions of the respective products, which belong to the product group, are continuous in the stage direction or in the column direction, the products are collectively placed, and thus the parallel request is satisfied. Further, for example, it is assumed that the reference display information indicates the arrangement request representing that a plurality of certain products should be placed adjacent to each other. In this case, the comparison unit 2040 determines whether or not the arrangement request is satisfied by determining whether or not the positions of the plurality of products are adjacent in the actual display information.
In the case where the reference display information indicates the applicable period, the comparison unit 2040 compares only the products whose applicable period includes a current time in the reference display information, among the products indicated in the actual display information, with the reference display information.
Each of the display 608, the position 610, the number of faces 612, the parallel request 614, and the arrangement request 616 indicates whether or not the actual display information matches the reference. For example, in the table 600, a case where the display 608 of a record of a certain product indicates “non-match” represents a case where the name of the product is indicated in the reference display information, and, on the other hand, the name of the product is not indicated in the actual display information. That is, the case indicates that a product to be displayed on the product shelf rack 20 is not displayed. In addition, for example, in the table 600, a case where the position 610 of the record of the certain product is “non-match” represents a case where the position of the product indicated in the actual display information does not match the reference indicated in the reference display information.
There are various methods for handling the comparison information. For example, the comparison unit 2040 stores the comparison information in the storage apparatus. In addition, for example, the comparison unit 2040 may generate image data or the like representing the result of the comparison using the comparison information, as described in an example embodiment which will be described later.
<<Restriction of Comparison Items>>
It is assumed that the reference display information indicates reference for a plurality of items such as the product name of the product to be displayed on the product shelf rack 20, the position of each product, and the number of faces of each product. In this case, the comparison unit 2040 may compare the actual display information with the reference display information for all items indicated in the reference display information, or may compare the actual display information with the reference display information for some items. In the latter case, for example, an item to be compared is specified by an input operation performed by the user.
Further, the comparison unit 2040 may compare the actual display information with the reference display information only for some products. For example, the comparison unit 2040 compares the actual display information and the reference display information only for the product of the determined manufacturer. In this case, for example, the determined manufacturer is specified by an input operation performed by the user.
A method for comparing the actual display information with the reference display information only for the determined manufacturer is not limited to the above-described method. For example, as described above, the reference display information may be prepared for each manufacturer, and only the reference display information of the determined manufacturer may be acquired by the comparison unit 2040.
<<Computation of Match Rate>>
The comparison unit 2040 may compute an index value representing the degree in which the display situation of the product represented by the actual display information matches the reference indicated in the reference display information. The index value is referred to as a match rate. For example, it is possible to define the match rate as a value acquired by dividing “the number of products, which are displayed on the product shelf rack 20 and which matches the reference indicated in the reference display information” by “a total number of products which should be displayed on the product shelf rack 20”. The following Equation (1) represents the definition. Here, “a certain product matches the reference” means that the information of the product indicated by the actual display information matches all the references (the position, the number of faces, and the like) indicated by the reference display information.
Further, the match rate may be computed not only for the individual product shelf rack 20 but also for an entire store. In this case, the match rate of the entire store is computed using the comparison information generated for each of the plurality of product shelves 20. Specifically, it is possible to define the match rate as a value acquired by dividing “the number of products which are displayed in the store and which match the reference indicated in the reference display information” by “the total number of products to be displayed in the store”. The following Equation (2) expresses the definition.
Hereinafter, the match rate (Equation (1)) computed for each shelf rack is referred to as a “match rate for each shelf rack”, and the match rate (Equation (2)) computed for the entire store is referred to as a “match rate for each store”. For example, in a case where one company manages a plurality of stores, it is possible to use the match rate for each store as one of the indexes for grasping an operation situation of each store. It can be said that, in a store having a high match rate for each store, the products are appropriately displayed according to the reference. On the other hand, there is a possibility that, in a store having a low match rate for each store, the products are not appropriately displayed.
Here, an opportunity, in which the information processing apparatus 2000 computes the match rate for each store, is arbitrary. For example, the information processing apparatus 2000 uses a fact that the comparison information is generated for all the product shelves 20 existing in the store as a trigger and computes the match rate for each store for the store.
It should be noted that, the match rate for each shelf rack and the match rate for each store may be computed not for all the displayed products but only for the products of the determined manufacturer. For example, it is assumed that the match rate for each shelf rack is computed for a certain manufacturer. In this case, the comparison unit 2040 computes a value acquired by dividing “the number of products that match the reference indicated in the reference display information among the products, which are placed on the product shelf rack 20, of the manufacturer” by “the total number of products of the manufacturer to be displayed on the product shelf rack 20” as the match rate for each shelf rack for the manufacturer. In addition, the comparison unit 2040 computes a value acquired by dividing the “number of products that match the reference indicated in the reference display information among the products of the manufacturer placed in the store” by “the number of products of the manufacturer to be displayed in the store” as the match rate for each store for the manufacturer.
In a case where, for the determined manufacturer, the match rate for each shelf rack or the match rate for each store is computed, the information processing apparatus 2000 acquires identification information (the manufacturer name, the manufacturer identifier, and the like) for determining the manufacturer. The identification information of the manufacturer is included in, for example, a request transmitted from the mobile terminal 3000.
<<Alert Using Match Rate>>
As described above, the information processing apparatus 2000 acquires the identifier of the product shelf rack 20 in order to identify the product shelf rack 20 included in the shelf rack image 12. Further, the information processing apparatus 2000 acquires the reference display information for the product shelf rack 20 determined by the identifier. Here, in a case where the information processing apparatus 2000 acquires an identifier of the product shelf rack 20, which is different from the product shelf rack 20 included in the shelf rack image 12, the information processing apparatus 2000 compares the display situation of the product shelf rack 20 included in the shelf rack image 12 with the reference display information for the different product shelf rack 20. Therefore, there is a high probability that the match rate for each shelf rack, which is computed in this case, is low.
Therefore, the information processing apparatus 2000 may be configured to determine whether or not the match rate for each shelf rack, which is computed by the comparison unit 2040, is equal to or less than a predetermined value, and to output an alert having a gist that there is a possibility that the identifier of the product shelf rack 20 is incorrect in a case where the match rate for each shelf rack is equal to or less than the predetermined value.
For example, it is assumed that the information processing apparatus 2000 receives a request, which includes the shelf rack image 12 and the identifier of the product shelf rack 20, from the mobile terminal 3000. In this case, in a case where the computed match rate for each shelf rack is equal to or larger than the predetermined value, the information processing apparatus 2000 transmits a message representing the above-described alert as a response with respect to the request. It is possible for the user, who receives the message, of the mobile terminal 3000 to take a measure of, for example, correcting transmission of the request by specifying the identifier of the product shelf rack 20.
Although a case where the alert is output in a situation in which there is a possibility that the identifier of the product shelf rack 20 is incorrect is exemplified in the above, the situation in which the alert is output is not limited to the situation. For example, in a case where at least any one of the match rate for each shelf rack and the match rate for each store, which are computed by the comparison unit 2040, is used and the match rate does not satisfy a predetermined condition, the information processing apparatus 2000 may output an alert representing a gist that an appropriate display according to the reference is not performed. For example, the predetermined condition is a condition that the match rate is equal to or larger than the predetermined value.
The information processing apparatus 2000 according to the second example embodiment includes a generation unit 2060. The generation unit 2060 generates a result image for the product shelf rack 20 using the comparison information generated for the product shelf rack 20. The result image of the product shelf rack 20 is image data represented products which do not match the reference indicated in the reference display information and products that match the reference indicated in the reference display information in a distinguishable aspect. The result image may be stored in the storage apparatus as an image file, or may be displayed on the display apparatus as screen data.
According to the information processing apparatus 2000 of the example embodiment, a result, which is acquired by comparing a display situation of the products on the product shelf rack 20 with the reference display information, is graphically represented in a form of the result image. Therefore, it is possible to intuitively and easily recognize a difference between the display situation of the products on the product shelf rack 20 and the reference for the display.
<Display Apparatus>
It is possible to use any display apparatus as the display apparatus on which the result image is displayed. For example, the display apparatus includes a display apparatus connected to the information processing apparatus 2000 or a display apparatus of the mobile terminal 3000. In addition, for example, the display apparatus may be a terminal installed in a staff room of a store where the product shelf rack 20 is installed, or a display apparatus connected to a terminal installed in an office of a company that manages the store. In addition, for example, it is assumed that a rounder or the like of a certain manufacturer uses the information processing apparatus 2000 in order to check whether or not a product of the manufacturer is correctly displayed on the product shelf rack 20. In this case, the display apparatus on which the result image is displayed may be a display apparatus connected to the terminal installed in the office or the like of the manufacturer. The terminal installed in the staff room or the office is, for example, a PC.
Here, in a case where the display apparatus, on which the result image is displayed, is not connected to the information processing apparatus 2000, the information processing apparatus 2000 transmits the result image to the terminal to which the display apparatus is connected. Therefore, it is necessary for the information processing apparatus 2000 to be able to determine a transmission destination of the result image.
There are various methods for determining the transmission destination of the result image. For example, information for determining the transmission destination of the result image is stored in the storage apparatus, which is accessible from the information processing apparatus 2000, in advance. In this case, the information processing apparatus 2000 determines the transmission destination of the result image by accessing the storage apparatus.
In addition, for example, in a case where the information processing apparatus 2000 performs a process according to the request transmitted from the mobile terminal 3000, the information processing apparatus 2000 transmits the result image to the transmission source of the request. The transmission destination of the result image may be included in the request. In this case, the information processing apparatus 2000 transmits the result image to the transmission destination indicated in the request.
<Result Image>
Specific content of the result image is various. Hereinafter, variations in the result image will be illustrated.
For example, the generation unit 2060 generates the result image by superimposing the superimposition data representing a result of the comparison of the actual display information with the reference display information on the shelf rack image 12.
The superimposition data may use a color instead of a pattern such as the dot pattern. For example, instead of the dot pattern, the oblique pattern, and the lattice pattern, respectively, a green color, a red color, and a yellow color are used.
Here, the superimposition data (the superimposition data 92 or the superimposition data 94 in
The product that matches the reference and the product that does not match the reference may be distinguished by a shape of the superimposition data instead of the pattern or the color of the superimposition data.
In addition, for example, a heat map may be used as the superimposition data.
The result image 80 may include a character string or the like representing the match rate for each shelf rack, in addition to the shelf rack image 12 on which the superimposition data representing the result of the comparison is superimposed.
The result image 80 may include a list of products displayed on the shelf rack image 12 in addition to the shelf rack image 12 on which the superimposition data representing the result of the comparison is superimposed.
Here, in a case where the result image is displayed on the display apparatus, the generation unit 2060 may change the display of the area 120 according to an input operation performed by the user on the product list 140. For example, it is assumed that the user selects a certain record in the product list 140. In this case, the comparison unit 2040 is configured to emphasize an image region corresponding to a product indicated by the selected record in the area 120.
In contrast, the generation unit 2060 may change the display of the product list 140 according to the input operation performed by the user on the product image displayed in the area 120. Specifically, in a case where the user selects an image of a certain product, the record of the product list 140 representing the product is emphasized.
<Method for Updating Display of Display Apparatus>
In a case where the display apparatus that displays the result image 80 is connected to a terminal other than the information processing apparatus 2000, an input operation with respect to the result image 80 is detected by the terminal. For example, in a case where the result image 80 is displayed on the display apparatus of the mobile terminal 3000, the input operation with respect to the result image 80 is detected by the mobile terminal 3000.
As described above, in a case where the input operation with respect to the result image 80 is detected by the terminal other than the information processing apparatus 2000, it is possible to use various methods as a method for updating display of the display apparatus according to the input operation. For example, a terminal that has detected the input operation with respect to the result image 80 transmits content of the input operation (for example, information relevant to the superimposition data that is a target of the input operation) to the information processing apparatus 2000. The information processing apparatus 2000 generates the result image 80 whose content is updated based on the received information, and transmits the result image 80 to the terminal. Then, the terminal displays the received result image 80 on the display apparatus. Through the series of flows, the result image 80 displayed on the display apparatus is updated.
In addition, for example, a terminal, such as the mobile terminal 3000, may have a function of updating the display of the display apparatus according to the input operation with respect to the result image 80. In this case, in a case where the terminal detects the input operation performed by the user, the terminal changes the result image 80 by itself without transmitting content of the input operation to the information processing apparatus 2000. In this case, an application for browsing the result image 80 is installed in the terminal in advance. For example, in a case of the mobile terminal 3000, the above-described application 100 has a function of browsing the result image 80.
<Case where Product that Cannot be Recognized from Shelf Rack Image 12 Exists>
There may be a case where it is not possible to recognize a part of the product included in the shelf rack image 12 because a part of the shelf rack image 12 is unclear. In this case, the actual display information may indicate a position where some undetermined products (unknown products) are placed.
In a case where the actual display information indicates a position of the unknown product, it is preferable that the result image 80 indicates the position of the unknown product. Specifically, the generation unit 2060 includes the superimposition data representing the unknown product in the result image 80.
For the unknown product, it is preferable to perform imaging with the camera 10 again. Therefore, for example, the superimposition data 110 may be realized as a button for calling a camera function. In this case, for example, the result image 80 is displayed on the display apparatus of the mobile terminal 3000 provided with the camera 10. In this case, the application 100 detects a fact that the user presses the superimposition data 110, or the like, and in response thereto activates a camera application that controls the camera 10. In a case where the user operates the camera application to cause the camera 10 to perform imaging, the application 100 transmits the generated shelf rack image 12 to the information processing apparatus 2000. The information processing apparatus 2000 generates the actual display information again using the acquired shelf rack image 12.
Here, in a case where a product that was the unknown product is included to be recognizable (clear) in the newly generated shelf rack image 12, the product name or the like of the product that was the unknown product is indicated in the newly generated actual display information. Therefore, the information processing apparatus 2000 generates the comparison information again by comparing the newly generated actual display information with the reference display information, and updates the result image 80. At this time, the information processing apparatus 2000 may generate the comparison information only for the product that is the unknown product in the previously generated actual display information, and may merge the comparison information with the previously generated comparison information.
It should be noted that, a method for using the superimposition data 110 representing the unknown product is not limited to a method for performing imaging by the camera 10 again. For example, a configuration may be used in which the superimposition data 110 includes information relevant to a product to be placed at the position of the unknown product. A display aspect of the information relevant to the product may use the same configuration as the information (correction information 150-1 and correction information 150-2) illustrated in
<Display of Information for Correcting Display of Product>
The result image 80 may include information representing the reference indicated in the reference display information (that is, information representing how to correct the display situation so as to match the reference) for the part that does not match the reference indicated in the reference display information. Hereinafter, the information is referred to as the correction information.
Here, in a case where the result image 80 is displayed on the display apparatus, the generation unit 2060 may include the correction information 150 in the result image 80 from the beginning, or may include the correction information 150 in the result image 80 at the timing when the input operation is added with respect to the superimposition data (the superimposition data 92 and the superimposition data 94) which indicates not matching the reference.
In addition, the generation unit 2060 may generate the screen data representing detailed information relevant to the product indicated in the correction information 150 according to a fact that the input operation is added to the correction information 150. The information relevant to the product includes, for example, an identifier of the product, a group to which the product belongs, a manufacturer of the product, and a place where the product is stored (a place of a shelf rack installed in a warehouse and a position of the product on the shelf rack). For example, in a case of the shortage of the product, it is necessary to bring the product from a storage location and replenish the product. Thus, the storage location of the product is displayed, and thus it is possible to easily perform an operation of replenishing the product.
Here, the correction information 150 may be displayed for all products that do not match the reference indicated in the reference display information, or may be displayed for some products. In the latter case, whether or not to display the correction information 150 for a product is determined based on a degree of importance of the product. In this case, the degree of importance is set for each product in advance.
For example, the generation unit 2060 generates the correction information 150 only for products whose degree of importance is equal to or larger than a predetermined value, and includes the correction information 150 in the result image 80. In addition, for example, the generation unit 2060 may generate the correction information 150 only for a predetermined number of products at a high degree of importance among the products that do not match the reference indicated in the reference display information, and include the correction information 150 in the result image 80.
[Example of Hardware Configuration]
Similar to the first example embodiment, a hardware configuration of the computer that realizes the information processing apparatus 2000 according to the second example embodiment is illustrated in, for example,
Hereinabove, although the example embodiments of the present invention are described with reference to the accompanying drawings, the example embodiments are examples of the present invention, and it is possible to use a combination of the respective example embodiment or to use various configurations other than the above-described configurations.
Although a part or all of the above-described example embodiments may be described as the supplementary below, the present invention is not limited thereto.
Number | Date | Country | Kind |
---|---|---|---|
2017-191790 | Sep 2017 | JP | national |
This application is a Continuation application of Ser. No. 17/986,038 filed Nov. 14, 2022, which is a Continuation application of Ser. No. 16/651,762, filed Mar. 27, 2020, which issued as U.S. Pat. No. 11,531,960, which is a National Stage of International Application No. PCT/JP2018/033600, filed Sep. 11, 2018, claiming priority to Japanese Patent Application No. 2017-191790, filed Sep. 29, 2017, the contents of all of which are incorporated herein by reference in their entirety.
Number | Date | Country | |
---|---|---|---|
Parent | 17986038 | Nov 2022 | US |
Child | 18228975 | US | |
Parent | 16651762 | Mar 2020 | US |
Child | 17986038 | US |