Patent Application
20230237687
The present invention relates to a product identification apparatus, a product identification method, and a program.
In a store where a product is sold, a position of a product on a product shelf, specifically, a shelf layout is important, since the shelf layout affects sales in the store. For example, Patent Documents 1 and 2 describe that, by processing a captured image of a product shelf, a product region image included in the image is determined, and a product is determined for each product region image. Further, Patent Document 2 also describes that skew correction or distortion correction is performed with respect to an image.
It is often a case that a plurality of product shelves are arranged in one store. Therefore, in order to efficiently manage a state of a shelf layout, it is desirable to divide a product shelf into a plurality of regions at a significant position, and determine a kind of a product for each region.
One example of an object of the present invention is to divide a product shelf into a plurality of regions at a significant position, and determine a kind of a product for each region.
The present invention provides a product identification apparatus including:
an acquisition unit that acquires an image of a shelf on which a product is placed;
an image processing unit that determines a position where continuity of the shelf is broken by processing the image, divides the shelf into a unit region by using the position, and also determines the product; and
a storage processing unit that causes a storage unit to store product identification information of the product located in the unit region, for each of the unit regions.
The present invention provides a product identification method including,
by a computer:
acquiring an image of a shelf on which a product is placed;
determining a position where continuity of the shelf is broken by processing the image, dividing the shelf into at least one unit region by using the position, and also determining the product; and
causing a storage unit to store product identification information of the product located in the unit region, for each of the unit regions.
The present invention provides a program causing a computer to include:
an acquisition function of acquiring an image of a shelf on which a product is placed;
an image processing function of determining a position where continuity of the shelf is broken by processing the image, dividing the shelf into at least one unit region by using the position, and also determining the product; and
a storage processing function of causing a storage unit to store product identification information of the product located in the unit region, for each of the unit regions.
The present invention enables to divide a product shelf into a plurality of regions at a significant position, and determine a kind of a product for each region.
The above-described object, the other objects, features, and advantages will become more apparent from suitable example embodiments described below and the following accompanying drawings.
Hereinafter, example embodiments according to the present invention are described with reference to the drawings. Note that, in all the drawings, a similar constituent element is indicated by a similar reference sign, and description thereof will not be repeated as appropriate.
The image capturing apparatus 10 is a portable apparatus. The image capturing apparatus 10 may be a communication apparatus with an image capturing function, such as a so-called smartphone. A user of the image capturing apparatus 10 divides the product shelf 40 into a plurality of regions, generates an image for each region, and transmits the image to an external apparatus, for example, the product identification apparatus 20. Further, the product identification apparatus 20 performs processing of determining a position of the product 50 with respect to each of a plurality of images.
Generally, it is often a case that a plurality of product shelves 40 are provided one over another in a vertical direction. Further, it is often a case that a vertical interval between the product shelves 40 is set according to a kind of the product 50 to be placed on the product shelf 40. Therefore, it is often a case that a height of the product shelf 40 becomes discontinuous at a position where a kind of the product 50 is changed. The product identification apparatus 20 determines a position where continuity of the product shelf 40 is broken by processing an image generated by the image capturing apparatus 10, and divides the product shelf 40 into a unit region by using the position. Further, the image capturing apparatus 10 determines the product 50 by processing the image, and causes a storage unit (e.g., a storage unit 240 to be described later) to store product identification information of the product 50 located in the unit region, for example, a JAN code, for each unit region.
In the example illustrated in
Further, the image processing unit 220 may perform distortion correction processing. A distortion to be corrected herein is, for example, a distortion due to an optical system (e.g., a lens) or the like of the image capturing apparatus 10. Further, in a case where an image includes a product shelf 40 having a curved edge surface, the image processing unit 220 may perform distortion correction in such a way that the product shelf 40 has a straight edge surface. In this case, the position of the product 50 is also corrected according to a correction amount of the edge surface of the product shelf 40.
Note that, as illustrated in
Further, in the example illustrated in
The bus 1010 is a data transmission path along which the processor 1020, the memory 1030, the storage device 1040, the input/output interface 1050, and the network interface 1060 mutually transmit and receive data. However, a method of mutually connecting to the processor 1020 and the like is not limited to bus connection.
The processor 1020 is a processor to be achieved by a central processing unit (CPU), a graphics processing unit (GPU), or the like.
The memory 1030 is a main storage apparatus to be achieved by a random access memory (RAM) or the like.
The storage device 1040 is an auxiliary storage apparatus to be achieved by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores a program module achieving each function (e.g., the acquisition unit 210, the image processing unit 220, and the storage processing unit 230) of the product identification apparatus 20. The processor 1020 achieves each function associated with the program module by reading each program module in the memory 1030 and executing each program module. Further, the storage device 1040 also functions as the storage unit 240.
The input/output interface 1050 is an interface for connecting the product identification apparatus 20 and various pieces of input/output equipment with each other.
The network interface 1060 is an interface for connecting the product identification apparatus 20 to a network. The network is, for example, a local area network (LAN) or a wide area network (WAN). A method of connecting the network interface 1060 to a network may be wireless connection, or may be wired connection. The product identification apparatus 20 may communicate with the image capturing apparatus 10 via the network interface 1060.
Subsequently, the image processing unit 220 of the product identification apparatus 20 determines a position where continuity of the product shelf 40 is broken by processing the image acquired in step S10 (step S20), and sets a boundary of a unit region at the position (step S30).
Further, the image processing unit 220 identifies a kind and a position of the product 50 by processing the image acquired in step S10 (step S40). Then, the storage processing unit 230 stores product identification information of the product 50 identified by the image processing unit 220, for each unit region set by the image processing unit 220 (step S50).
Note that, the image processing unit 220 may perform step S40 prior to step S20 and step S30, or may perform step S40 concurrently with step S20 and step S30.
As described above, according to the present example embodiment, the product identification apparatus 20 determines a position where continuity of the product shelf 40 is broken, and sets a unit region by using the position. Generally, it is often a case that a height of a product shelf 40 becomes discontinuous at a position where a kind of the product 50 is changed. Therefore, setting a unit region, and storing product identification information of the product 50 for each unit region, as the product identification apparatus 20, enables to efficiently manage a state of a shelf layout.
Further, an image processing unit 220 of the product identification apparatus 20 generates a stitched image in which a plurality of images (e.g., a first image and a second image) generated by the image capturing apparatus 10 are stitched in a horizontal direction, and sets the sequential image as a processing target described in the first example embodiment.
According to the present example embodiment, even when the product shelf 40 is long in a horizontal direction, it is possible to set a unit region in the product shelf 40, and store product identification information of a product 50 for each unit region.
As described above, while the example embodiments according to the present invention have been described with reference to the drawings, these example embodiments are an example of the present invention, and various configurations other than the above can also be adopted.
Further, in a plurality of flowcharts used in the above description, a plurality of processes (pieces of processing) are described in order, but an order of execution of processes to be executed in each example embodiment is not limited to the order of description. In each example embodiment, the illustrated order of processes can be changed within a range that does not adversely affect a content. Further, the above-described example embodiments can be combined, as far as contents do not conflict with each other.
A part or all of the above-described example embodiments may also be described as the following supplementary notes, but is not limited to the following.
1. A product identification apparatus including:
an acquisition unit that acquires an image of a shelf on which a product is placed;
an image processing unit that determines a position where continuity of the shelf is broken by processing the image, divides the shelf into a unit region by using the position, and also determines the product; and
a storage processing unit that causes a storage unit to store product identification information of the product located in the unit region, for each of the unit regions.
2. The product identification apparatus according to supplementary note 1, wherein
the image includes a plurality of the shelves with different heights,
the unit region is set by being divided in a horizontal direction, and
the image processing unit determines a position where continuity is broken, for each of the plurality of shelves, and sets a boundary that defines the unit region at the position where continuity is broken in at least one of the shelves.
3. The product identification apparatus according to supplementary note 2, wherein
the plurality of shelves are separately captured into a plurality of images aligned in a horizontal direction, and
the image processing unit generates a stitched image in which the plurality of images are stitched, and sets the unit region by using the stitched image.
4. The product identification apparatus according to supplementary note 2 or 3, wherein
the image processing unit determines a number of shelves in the unit region, for each of the unit regions, and
the storage processing unit causes the storage unit to store a number of shelves in the unit region, for each of the unit regions.
5. A product identification method including,
performing by a computer:
acquisition processing of acquiring an image of a shelf on which a product is placed;
image processing of determining a position where continuity of the shelf is broken by processing the image, dividing the shelf into at least one unit region by using the position, and also determining the product; and
storage processing of causing a storage unit to store product identification information of the product located in the unit region, for each of the unit regions.
6. The product identification method according to supplementary note 5, wherein
the image includes a plurality of the shelves with different heights, and
the unit region is set by being divided in a horizontal direction,
the product identification method further including,
by the computer,
in the image processing, determining a position where continuity is broken, for each of the plurality of shelves, and setting a boundary that defines the unit region at the position where continuity is broken in at least one of the shelves.
7. The product identification method according to supplementary note 6, wherein
the plurality of shelves are separately captured into a plurality of images aligned in a horizontal direction,
the product identification method further including,
by the computer,
in the image processing, generating a stitched image in which the plurality of images are stitched, and setting the unit region by using the stitched image.
8. The product identification method according to supplementary note 6 or 7, further including,
by the computer:
in the image processing, determining a number of shelves in the unit region, for each of the unit regions; and,
in the storage processing, causing the storage unit to store a number of shelves in the unit region, for each of the unit regions.
9. A program causing a computer to include:
an acquisition function of acquiring an image of a shelf on which a product is placed;
an image processing function of determining a position where continuity of the shelf is broken by processing the image, dividing the shelf into at least one unit region by using the position, and also determining the product; and
a storage processing function of causing a storage unit to store product identification information of the product located in the unit region, for each of the unit regions.
10. The program according to supplementary note 9, wherein
the image includes a plurality of the shelves with different heights,
the unit region is set by being divided in a horizontal direction, and
the image processing function determines a position where continuity is broken, for each of the plurality of shelves, and sets a boundary that defines the unit region at the position where continuity is broken in at least one of the shelves.
11. The program according to supplementary note 10, wherein
the plurality of shelves are separately captured into a plurality of images aligned in a horizontal direction, and
the image processing function generates a stitched image in which the plurality of images are stitched, and sets the unit region by using the stitched image.
12. The program according to supplementary note 10 or 11, wherein
the image processing function determines a number of shelves in the unit region, for each of the unit regions, and
the storage processing function causes the storage unit to store a number of shelves in the unit region, for each of the unit regions.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2020/019258 | 5/14/2020 | WO |
