Patent Application
20210374476
The present invention relates to a method for identifying authenticity of an object more particularly to the according to preamble of claim 1. The present invention further relates to a system for identifying authenticity of an object and more particularly to a system according to preamble of the claim 15.
Counterfeit objects or products, such as pharmaceutical product, are an increasing problem and battle against counterfeits is becoming increasingly difficult. In many cases the counterfeit products, such as medicines, may subject health of users under danger. There exist many conventional methods for marking original products or goods such that they could be identified as authentic original products. These conventional comprise hologram stickers, special seals and tracking numbers which may be placed on packaging of products. However, identifying these conventional markers as authentic requires special skills preventing utilizing them in large scale. There also exist digital methods for marking original products or goods such that they could be identified as authentic original products. These conventional digital methods comprise RFID tags or the like means for digital tracking of products. However, these digital methods are expensive to produce and require extra work when utilized in large volumes.
One of the problems associated with the prior art identification methods and means is they always require extra work phases carried out during production process or packing process. Furthermore, the prior art methods also require adding something, such as stickers, tags or seals, to the product. One further disadvantage is that these conventional methods are usually carried out by an external producer or service provider. Therefore, the brand owner or company which actually owns the product cannot control and execute the authenticity identification from start to finish by itself. Furthermore, the consumers and users of the products cannot determine by themselves if the product is authentic or not, based on these conventional methods. Therefore, the conventional methods do not prevent consumers from using counterfeit objects or products.
An aim of the present invention is to provide a method and system so as to solve or at least alleviate the prior art disadvantages.
The aims of the invention are achieved with a method for identifying authenticity of an which is characterized by what is stated in claim 1. The aims of the invention are further achieved with a system for identifying authenticity of an object which is characterized by what is stated in claim 15.
The preferred embodiments of the invention are disclosed in the dependent claims.
The invention is based on the idea of providing a method for identifying authenticity of an object with a user device having a processor and a memory and which the method comprises the steps of:
a) providing in an identification server (20), a reference identification algorithm trained with reference image data of an original object;
b) receiving, in the identification server (20), one or more target images of the object from the user device over a communication network;
c) automatically processing, by utilizing instructions stored in the memory and executed by the processor of the identification server, the one or more target images with the reference identification algorithm trained with the reference image data of the original object;
d) automatically generating, by utilizing the reference identification algorithm and instructions stored in the memory and executed by the processor of the identification server, an authenticity identification output in the identification server; and
e) automatically sending, by utilizing instructions stored in the memory and executed by the processor of the identification server, an authenticity identification response based on the authenticity identification output from the identification server to the user device.
The step a) of providing a reference identification algorithm may comprise providing, in the identification server, two or more feature identification algorithms trained with the reference image data of the original object as the reference identification algorithm for identifying separate features of the original object in the identification server and the processing the one or more target images with reference identification algorithm trained with the reference image data of the original object comprises processing, by utilizing instructions stored in the memory and executed by the processor of the identification server, the one or more target images with the two or more feature identification algorithms trained with the reference image data of the original object.
The two or more feature identification algorithm provides the possibility to identify multiple features of the object presented in the one or more target images. The features can be for example a logo, a text, a colour, a shape, a structure or dimension. This improves the authenticity identification process to become faster and more reliable.
The method may further comprise processing, by utilizing instructions stored in the memory and executed by the processor of the identification server, the one or more target images with at least two of the two or more feature identification algorithms in parallel; generating, by utilizing at least the two or more feature the reference identification algorithms and instructions stored in the memory and executed by the processor of the identification server, an output from each of the two or more feature identification algorithms; and sending, by utilizing instructions stored in the memory and executed by the processor of the identification server, one common authenticity identification response based on the outputs of the two or more feature identification algorithms from the identification server to the user device.
In the step of processing the one or more target images with two or more feature identification algorithms in parallel the one or more target images are processed with different features in different feature identification algorithms and all these feature identification algorithms generate an output such that there are at least two outputs expressing the results of different feature identifications and these outputs are provided as a common output such that the identification server sends a common authenticity identification response to the user device. This step has the benefit that different features can be processed in the feature identification algorithms simultaneously which gives a fast response from the identification server to the user of the user device to inform the user the authenticity of the object.
The method may further comprise processing, by utilizing instructions stored in the memory and executed by the processor of the identification server, the one or more target images with one or more of the two or more feature identification algorithms in series; generating, by utilizing the reference identification algorithm and instructions stored in the memory and executed by the processor of the identification server, an output from the one or more of the two or more feature identification algorithms; and sending, by utilizing instructions stored in the memory and executed by the processor of the identification server, one authenticity identification response based on the outputs of the one or more features identification algorithms from the identification server to the user device.
In the step of processing the one or more target images with two or more feature identification algorithms in series the one or more target images are processed with one feature at a time such that if the first feature is identified as authentic the process continues to the next feature identification algorithm and provides an output from that algorithm. The features are identified one by one until one feature is identified as a non-genuine or until all features are identified as authentic.
The processing the target image with one of the two or more feature identification algorithms may comprise comparing, by utilizing instructions stored in the memory and executed by the processor of the identification server, one feature of the one or more target images with the reference image data utilizing the one of the two or more feature identification algorithms; generating, by utilizing instructions stored in the memory and executed by the processor of the identification server, a first feature authenticity identification output in the identification server; and based on the first feature authenticity identification output:
The above-mentioned steps provide the process for the feature identification of the one or more target images. In those steps features are compared one feature at a time with the reference image data of the original object utilizing the feature identification algorithms and if the feature is within a preselected range, which can be for example a certain percentage range representing the estimate of authenticity of the object represented in the one or more target images, the process continues to the next feature which is compared to the reference image data of the original object utilizing the feature identification algorithms. The certain percentage range representing the estimate of authenticity of the object can be for example a range of 80-100% meaning that the certainty of the authenticity of the feature in the object is within that range. The features are processed until there exists a feature which is outside the preselected range, i.e. most certainly not authentic or until all the features are processed and found to be in the preselected range.
The one or more target images may be provided with additional data such that the additional data is sent together with the one or more target images to the identification server. The additional data can be for example GPS-coordinates or user information which can be beneficial when tracking counterfeit hot-spots around the world. The one or more target images and additional data may be stored in a target image database as a target image profile in one or more storage devices. The one or more storage device comprising the target image database may be provided in the identification server or separately from the identification server but connected to the identification server in another server.
The method may further comprise in step b) creating, by utilizing instructions stored in the memory and executed by the processor of the identification server, a one or more standardized target images by image pre-processing the one or more target images. The image pre-processing may comprise cropping, by utilizing instructions stored in the memory and executed by the processor of the identification server, the one or more target images; or rotating, by utilizing instructions stored in the memory and executed by the processor of the identification server, the one or more target images; or pre-processing, by utilizing instructions stored in the memory and executed by the processor of the identification server, the one or more target images by digital filtering or affine transformation. In an alternative embodiment, the one or more standardized target images may be provided by creating or automatically creating, by utilizing instructions stored in the user device memory and executed by the user device processor of the user device, a one or more standardized target images by image pre-processing the one or more target images in the user device. Accordingly, the standardization of the one or more target images may be carried out in the user device or in the identification server.
By standardizing the one or more target images, the one or more target images are modified such that the identification of the authenticity of the object is most advantageous with the reference identification algorithm trained with the reference image data of the original object.
The method may further comprise sending, by utilizing instructions stored in the memory of the user device and executed by the processor of the user device, the one or more target images of the object from the user device to the identification server. In one embodiment, the sending is carried out with a user interface in the user device on request by the user of the user device. In the step of sending the one or more target images may be already provided in the memory of the user device.
The method may alternatively comprise taking, by utilizing image taking instructions stored in the memory and executed by the processor in the user device, the one or more target images of the object with imaging device in the user device on request by the user of the user device. The image taking instructions may be provided in the user interface of the user device, which the image taking instructions guide the user of the user device for example to adjust lighting conditions or to adjust an image taking area to match the object such that the object fills the image taking area ideally. The imaging device is preferably an integral camera in the user device.
With the step of taking the one or more target images the user of the user device utilizes image taking instructions in which it is instructed to take the one or more target image of the object such that the object will be shown ideally in the one or more target image so that it is most suitable for processing with the reference identification algorithm. An example of utilizing image taking instructions is that the user is presented with a clear frame in the user interface in which the object must fully be inside of.
Alternatively, the method may comprise taking, by utilizing image taking instructions stored in the memory and executed by the processor in the user device, the one or more target image of the object with imaging device in the user device on request by the user of the user device and sending, by utilizing instructions stored in the memory of the user device and executed by the processor of the user device, the one or more target image of the object from the user device to the identification server.
The method may further comprise storing, by utilizing image taking instructions stored in the memory and executed by the processor in the user device, the one or more target image of the object in a target image database in the identification server or in one or more storage devices.
The method may further comprise providing, by utilizing instructions stored in the memory and executed by the processor in the user device, an identifier for the object in the user interface on request by the user of the user device; sending, by utilizing instructions stored in the memory of the user device and executed by the processor of the user device, the one or more target image of the object from the user device to the identification server on request by a user of the user device; connecting, by utilizing instructions stored in the memory and executed by the processor of the identification server, the one or more target image based on the identifier to a object profile in an original object database in the identification server, the original object database being provided in one or more storage devices; and selecting, by utilizing instructions stored in the memory and executed by the processor of the identification server, the reference identification algorithm connected to the object profile.
The providing an identifier means selecting or naming the object from which the one or more target image is taken by the user of the user device such that when the one or more target image together with the identifier is sent to the identification server and the one or more target image based on the identifier is connected to an object profile in the original object database and the reference algorithm is selected based on the identifier which narrows down the scope of the reference data of the original objects from which the reference identification algorithm is generated so that the processing the one or more target image with the reference identification algorithm becomes faster.
The method may further comprise storing, by utilizing instructions stored in the memory and executed by the processor in the user device, the authenticity identification response in a target image database in one or more storage devices in the identification server. Alternatively, the method may further comprise storing, by utilizing instructions stored in the memory and executed by the processor in the user device, the authenticity identification response in the target image database in one or more storage devices connected to the identification server over a communication network.
The method of the present invention may further comprise:
In the case of receiving, in the identification server, the original object video, the video is split into the separate original images. However, in the case of receiving one or more original images, the splitting step is omitted. It should be noted that the basic identification algorithm may be trained with only one original object image or with two or more original object images.
According to the above mentioned, the method of the present invention enables an administrator, for example a brand owner, upload one or more original object images or an original object video to the identification server and form a reference identification algorithm which may be further utilized for identifying counterfeit objects from original object.
In one embodiment of the present invention the step of creating object profile comprises:
In this embodiment, the administrator may create and store a new object profile to the original object database and input object information and data of the original object to the object profile. The reference identification algorithm is connected to the object profile such that each specific reference algorithm is connected to a specific object profile.
In one embodiment, the receiving of the one or more original object images or the original object video as the reference image data to the identification server comprises:
Accordingly, the administrator may take the one or more original object images or the original object video of the original object by itself and upload it to the identification server using the administrator device, such as mobile phone, tablet computer or the like.
In one embodiment of the present invention the splitting of the original object video into the several separate original images comprises splitting, by utilizing instructions stored in a memory and executed by a processor of the identification server, the original object video into video frames to create the several separate original images. This enables splitting the original object video into separate original images such that every frame of the original object video forms a separate original image from each of the video frames maximizing the number of separate original images.
In one embodiment, the method comprises creating, by utilizing instructions stored in the memory and executed by the processor of the identification server, standardized separate original images by image pre-processing the one or more original object images, or the several original images generated from the original object video. Creating the standardized several original images may comprise:
In a preferred embodiment, standardizing the one or more target images and standardizing the one or more original object images may be carried out in similar, corresponding or identical manner.
In one embodiment, the providing of the basic identification algorithm comprises providing the basic identification algorithm from an external algorithm service, or providing the basic identification algorithm from external algorithm service comprising a machine learning algorithm such as artificial neural network, such as a convolutional neural network or capsule network. In an alternative embodiment, the providing of the basic identification algorithm comprises providing the basic identification algorithm from algorithm service provided in connection with the identification server, or providing the basic identification algorithm from algorithm service provided in connection with the identification server, the algorithm service comprising basic machine learning algorithm, or basic artificial neural network. Accordingly, the algorithm service may be an external service or internal service provided to the identification server. The algorithm service enables creating reference identification algorithm by training the basic identification algorithm with the one or more standardized original images. This may be carried out by training the basic machine learning algorithm, or the basic artificial neural network with the one or more standardized original object images.
In one embodiment of the present invention, the applying the one or more standardized original object images as input data to the basic identification algorithm, machine learning algorithm, comprises, or generating, by utilizing instructions of the algorithm service, the two or more feature reference identification algorithms for identifying separate features of the original object, or generating, by utilizing instructions of the algorithm service, two or more separate feature reference identification algorithms for identifying separate features of the original object, or generating, by utilizing instructions of the algorithm service, the reference identification algorithm comprising two or more feature reference identification algorithms for identifying separate features of the original object. The reference identification algorithm may comprise or be a combination of two or more feature identification algorithms or consist of two or more separate feature identification algorithms. Each of the feature identification algorithms may be provided and trained to identify one specific feature or characteristic of an object. Accordingly, the identification may be targeted to two or more specific features of the original object based on which the identification of authenticity may be determined, as described above.
In one embodiment, the generating, by utilizing instructions of the algorithm service, of the reference identification algorithm or the two or more feature reference identification algorithms may comprise generating, by utilizing instructions of the algorithm service, a shape reference identification algorithm for identifying shape of the object. This algorithm may be provided to identify or recognize the outer shape or contour of the object such that the shape of the object may be identified.
In an alternative embodiment, the generating of the identification algorithm or the two or more feature reference identification algorithms may comprise generating a visual symbol reference identification algorithm for identifying visual symbols on the object. This algorithm may be provided to identify or recognize graphical symbols or pictures on the surface of the object, such as logos.
In a yet alternative embodiment, the generating of the identification algorithm or the two or more feature reference identification algorithms may comprise generating an optical character recognition reference identification algorithm for identifying shape of the product. This algorithm may be provided to identify or recognize text provided on the surface of the object.
In one embodiment, the method further comprises
In one embodiment, the method further comprises creating, with an administrator device, at least two object sub-profiles on a request by the administer for the original object, the at least two object sub-profiles representing different sides of the original object, or creating, with an administrator device, at least two object sub-profiles on a request by the administer for the original object, the at least two object sub-profiles representing different parts of the original object, such as outer packaging, inner packaging and a product. The inner packaging may be for example blister pack for medicine pills or the like, and the object may be a medicine pill. These all may be considered as objects. Accordingly, there may be different identification algorithms for different sides or parts of the object such that identifying the authenticity may be carried out based on images of different sides or different parts of the object to be identified. Alternative, the identifying may be based on combination of the separate identification algorithms of the different sub-profiles.
The present invention is further based on the idea of providing an authenticity identifying system for identifying authenticity of an object, which the system comprises:
The authenticity identifying system may further comprise the user interface in the user device being operable to taking the one or more target images; or the user device may be operable to taking the one or more target images.
The authenticity identifying system may further comprise the user interface in the user device being operable to displaying or indicating the authenticity identification response; or the user device being operable to displaying or indicating the authenticity identification response.
The authenticity identifying system may further comprise the user interface in the user device being operable to providing an identifier for the object and/or user data; or the user device being operable to providing an identifier for the object and/or user data.
The system for identifying authenticity of an object may further comprise:
According to the above mentioned, the system of the present invention enables the brand owner or the company to set up authenticity identification for own original objects.
In one embodiment, the administrator interface in the administrator device may be operable to taking the one or more original object images or the original object video on a request by the administrator. Alternatively, the administrator device may be operable to taking the one or more original object images or the original object video on a request by the administrator. This allows the administrator or the brand owner to take the one or more original object images or the original object video easily, for example using a digital camera or user device such as mobile phone.
It should be noted, that in the case of taking the one or more original object images instead of the original object video, the splitting step may be omitted.
In one embodiment, administrator interface in the administrator device may be operable to sending the one or more original object images or original object video on a request by the administrator to the identification server over the communication network. Therefore, the administrator may send or upload the one or more original object images or the original object video to the identification server and set up the identification from the administrator device, such as a mobile phone or laptop computer.
In one embodiment, the administrator interface in the administrator device may be operable to creating at least two object sub-profile on a request by an administer for the original object, the at least two sub-profiles representing different sides or different parts of the original object. Accordingly, the administrator may create different sub-profiles for example for different parts or sides of the original object. This means, that the administrator may create a first sub-profile for the front side of the original object and back side of the original object. Alternative, the administrator may create for example a first sub-profile for outer packaging of the original object, a second sub-profile for inner packaging of the original object and a third sub-profile for the original object itself.
An advantage of the invention is that the method and system allows identifying authenticity of an object without adding any excess tags, stickers or visual symbols to the original object. The invention allows brand owners to manage counterfeits without any special knowledge of other technologies for identifying original objects. Furthermore, the method and system enables the brand owner or the company to fully control the counterfeit process by itself such that new products may be provided to the system and removed from the system at any point of time. Furthermore, the method and system allows brand owner to receive valuable information about counterfeit products when users upload images to the identification server for identifying the authenticity. The method and system is further very easy and simple to use and provides information in real time.
The invention is described in detail by means of specific embodiments with reference to the enclosed drawings, in which
In the following the present invention is described in detail referring to the accompanying drawings which also are included to the detailed description. It should be noted that the detailed description together with drawings show example embodiments for carrying out the present invention, but the present invention is not limited to detailed example embodiments. Furthermore, the different embodiment, elements, method steps, structural features described may be combined and changed without departing from the scope of the appended claims. The detailed description provides enough information for skilled person to utilize and practise the present invention.
In the context of this application the same reference numerals in different figures denote the same features. The term “user” refers to a first user of the method and system willing to identify if a pharmaceutical product is authentic or not. Therefore, the user means for example a buyer or consumer of the product in the context of the present application. The term “administrator” refers to a second user of the system and method willing to set up system or method for identifying authenticity of the product. Therefore, the administrator means for example brand owner or company producing the product or product. The term “product” refers to any physical pharmaceutical products, packaging of pharmaceutical products, medicines, such as pills, or combinations or parts thereof.
In the following detailed description and examples, the original object and object to be identified is a pharmaceutical product. Therefore, the object in the context of this application may be any physical object, such as pharmaceutical product, outer packaging, inner packaging, a pill or the like. Furthermore, the object may also be a bag, electronic device, art, painting, or any other physical product.
Further, the basic identification algorithm may be a machine learning algorithm such as an artificial neural network, or convolutional neural network. Neural networks are preferable, as they are able to process mixed data Flexible, use of mixed or heterogeneous data, such text, numeric and image data. Further they may be trained fast and efficiently. The reference identification algorithm may therefore be trained machine learning algorithm, such as trained artificial neural network.
In an alternative embodiment, at least one storage device 28, may be connected to the communication network 100, as in
In one embodiment, the administrator device 30 may also comprise a camera or imaging device 34 for taking a video or image. Furthermore, in some embodiments the administrator device 30 may also comprise a speaker and/or microphone 33. It should be noted, that in some embodiments the imaging device 34 may be integral to the administrator device 30, but in some other embodiments the imaging device may be a separate device, such as a separate digital camera.
The administrator device 30 may be same or different kind of device as the user device 10. In one embodiment, the administrator device 30 may be mobile communication device, such as mobile phone or a tablet computer suitable for performing web browsing or capable of accessing and interacting with the identification server 20. However, the user device may also be personal digital assistant, laptop computer, desktop computer, thin client, electronic notebook or any other such device having a display 31 and administrator interface 32 and being suitable for performing web browsing or capable of accessing and interacting with the identification server 20. In a preferred embodiment, the user device is a mobile device, meaning mobile phone, tablet computer or laptop computer.
The administrator interface 32 associated with the administrator device 30 may comprise any suitable interface 32 for a human user. In one embodiment, the administrator interface 32 may be graphical interface displayed on the display 31 of the administrator device 30. The administrator interface 32 may be accessible by the administrator with an input device 38 such as touchscreen, keyboard, mouse, touchpad, keypad, trackball or any other suitable hand operated input device or some other kind of input device such as voice operable user device or human gesture, such as hand or eye, gesture detecting input device. The input device 38 may be configured to receive input from the administrator. The administrator interface 32 (generally API, Application Programmable Interface) may be provided in conjunction with the system and method of the present invention in order to enable the administrator to interact with the identification server 20. It should be noted that the system and method may comprise more than one different kinds of administrator interfaces 32. Furthermore, the administrator device 30 may comprise an administrator application, such as a software application, stored in the administrator device memory 35, or administrator device storages 37, and executed with the administrator device processors 36.
The display 31 of the administrator device 30 may be any known display component, such as touch screen, computer display, projector or special display device allowing the administrator to display and operate the administrator interface 32. The display 31 may further comprise video graphics adapter card, liquid crystal display (LCD) component, light emitting diode (LED) element, or any other device or element capable of presenting the administrator interface 32.
In one embodiment, the user device 10 may also comprise a camera or imaging device 14 for taking a video or image. Furthermore, in some embodiments the user device 10 may also comprise a speaker and/or microphone 13.
The user device 10 may be similar to the administrator device 30, or it may be different, for example one of the devices mentioned in connection with the administrator device 30. In a preferred embodiment, the user device may be mobile phone. The user device 10 may be any device having a display 11 and user interface 12 and being suitable for performing web browsing or capable of accessing and interacting with the identification server 20. Furthermore, it should be noted that different user and administrator devices 10, 30 may be used for utilizing the method and system, of the present invention.
The user interface 12 associated with the user device 10 may comprise any suitable interface 12 for a human user. In one embodiment, the user interface 12 may be graphical user interface displayed on the display 11 of the user device 10. The user interface 12 may be accessible by the user with an input device 38 such as touchscreen 11 or any other input device as mentioned in connection with the administrator device 30. The input device 11 may be configured to receive input from the user. The user interface 12 (generally API, Application Programmable Interface) may be provided in conjunction with the system and method of the present invention in order to enable the user to interact with the identification server 20. It should be noted that the system and method may comprise more than one different kinds of user interfaces 12. Furthermore, the user device 10 may comprise a user application, such as a software application, stored in the user device memory 15, or user device storages 17, and executed with the user device processors 16.
The display 11 of the user device 10 may be any known display component, including the ones mentioned in connection with the administrator device 30, allowing the user to display and operate the user interface 12. The display 11 may further comprise video graphics adapter card, liquid crystal display (LCD) component, light emitting diode (LED) element, or any other device or element capable of presenting the user interface 12.
In one embodiment, the identification server 20 comprises one server and in an alternative embodiment the may comprise more than one server or other equipment, each performing different or same functions in order to receive and communicate information to the user devices 10 and administrator devices 30. The identification server 20 may comprise software and/or algorithms to achieve the operations for processing, communicating, delivering, gathering, uploading, maintaining and/or generally managing data. The software and/or algorithms may be stored on the memory 24 of the identification server 20. Alternatively, the mentioned operations and techniques may be achieved by any suitable hardware, component, device, application specific integrated circuit (ASIC), additional software, field programmable gate array (FPGA), server, processor, algorithm, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or any other suitable product that is operable to facilitate such operations.
The memory 24 of the identification server 20 may be a non-transitory computer-readable storage medium or a computer-readable storage device. In some embodiments the memory 24 may be a temporary memory, meaning that a primary purpose of memory 24 may not be long-term storage. Memory 24 may also refer to a volatile memory, meaning that memory 24 does not maintain stored contents when memory 24 is not receiving power. Examples of volatile memories include random access memories (RAM), dynamic random access memories (DRAM), static random access memories (SRAM), and other forms of volatile memories known in the art. In some examples, memory 24 is used to store program instructions for execution by the processors 22. Memory 24, in one embodiment, is used by software (e.g., an operating system) or applications, such as a software, firmware, or middleware. The memory 24 may comprise for example operating system or software application comprising at least part of the instructions for executing the method.
The above mentioned description of memory 24 may also concern memory 3z of the administrator device 30 and/or the memory 15 of the user device 10.
The identification server 20 may further comprise one or more storage devices 28, 29, as shown in
The storage devices 28, 29 may be provided in connection with the identification server 20 or the identification server 20 may comprise the storage devices 28, 29, as shown in
The communication network 100 may comprise one or more communication networks which may comprise, but is not limited to: the Internet, intranet, local area network (LAN), wide area network (WAN), metropolitan area network (MAN), cellular phone network (e.g. Global System for Mobile (GSM) communications network, packet switching communication network, circuit switching communication network), virtual private network, Bluetooth radio, an IEEE 802.11-based radio frequency network (e.g. CR)), or any other appropriate architecture or system that facilitates communications in a network or a combination of any networks or systems described.
The identification server 20 may further be provided with at least one network interface (not shown), such as application program interface (API). The network interface may be utilized to communicate with external devices and servers via one or more communication networks 100, such as one or more wired, wireless, or optical networks comprising, for example, the Internet, intranet, LAN, WAN, cellular phone networks, Bluetooth radio, an IEEE 802.11-based radio frequency network, and so forth. The network interface may be a network interface card, such as an Ethernet card, an optical transceiver, a radio frequency transceiver, or any other type of device that can send and receive information. Other examples of such network interfaces may comprise Bluetooth®, 3G, 4G, and WiFi® radios in mobile computing devices as well as USE. The identification server 20 may further also comprise power supply, housing, communication bus and other physical components.
The operating system may interact with the above described components of the identification server 20 and devices connected to the identification server 20 directly or via application(s) or instructions stored in the memory 24.
The identification server 20 may further comprise one or more additional services 26 providing additional information and data to the method according to the present invention, as shown in
As shown in
In one embodiment, the additional service 26 is an algorithm service operable to providing basic identification algorithm for the identification method of the present invention. The algorithm service may comprise artificial neural network, such as a convolutional neural network or capsule network, operable to providing the basic identification algorithm and generating a reference identification algorithm from input image data. In one specific example the algorithm service comprises publicly available machine learning library or the like.
As shown in
In some embodiments, the administrator interface 32 may instruct in taking the original product video. Accordingly, the administrator device 30 or the administrator interface 32 may instruct, by utilizing instructions stored in the administrator device memory 35 and executed by the administrator device processor 36, in taking the original product video. The administrator interface 32 in the administrator device 30 of the system 1 may be operable to instruct the administrator in taking the original product video.
In one embodiment, the administrator device 30 or the administrator interface 32 may instruct, by utilizing instructions stored in the administrator device memory 35 and executed by the administrator device processor 36, in taking the original product video on a predetermined angle or in predetermined angles. The administrator device 30 may comprise one or more accelerometers for determining the position or angle of the administrator device 30 or the camera 34. Accordingly, the accelerometer may be operable to determining or calculating position or posture of the administrator device 30, for example by utilizing instructions stored in the administrator device memory 35 and executed by the administrator device processor 36 of the administrator device 30. Accordingly, the administrator device 30 or the administrator interface 32 may instruct, by utilizing instructions stored in the administrator device memory 35 and executed by the administrator device processor 36, in taking the original product video on a predetermined angle or in predetermined angles based on the calculation or determination of the position or posture of the administrator device 30 with the one or more accelerometers.
In one embodiment, the administrator device 30 or the administrator interface 32 may instruct, by utilizing instructions stored in the administrator device memory 35 and executed by the administrator device processor 36, placing the original product on a horizontal plane or surface before taking the original product video. Then, the administrator device 30 or the administrator interface 32 may instruct taking the original product video in predetermined angle or angles.
According to the above mentioned, the original product video may be instructed to be taken in one angle or turning the camera or the administrator device 30 in different angles during taking the original product video.
In one embodiment, the administrator interface 32 in the administrator device 30 is operable in calculating, by utilizing instructions stored in the administrator device memory 35 and executed by the administrator device processor 36 and the accelerometer, the position or posture of the administrator device 30 or the camera during taking the original product video. The calculated position or posture of the administrator device 30 or the camera during taking the original product video may be send to the identification server 20 together with the original product video, or separately. The calculated position or posture of the administrator device 30 or the camera may further be stored to the product profile or provided as input data to the basic identification algorithm. In this embodiment, the administrator device 30 or the administrator interface 32 may instruct, by utilizing instructions stored in the administrator device memory 35 and executed by the administrator device processor 36, placing the original product on a horizontal plane or surface before taking the original product video. Then, the position or posture of the administrator device 30 or the camera is calculated during taking the original product video.
The method of the present invention further comprises step 350 of receiving an original product video as reference image data of the original product to the identification server 20 over the communication network 100.
One embodiment of step 350 is shown in
In an alternative embodiment, the step 352 may be replaced by uploading the original product video from an external source (not shown) to the administrator device, or taking the original product video with a separate imaging device (not shown) and uploading the original product video from an external source (not shown) to the administrator device, and/or uploading the original product video from the one or more administrator device storages 37 to the administrator interface 32 in the administrator device 30.
The step 350 may further comprise step 354 of sending or uploading the original product video to the identification server 20 via the communication network 100, on request of the administrator in the administrator interface 32. The sending may also be carried out automatically, by utilizing image taking instructions stored in the administrator device memory 35 and executed by the administrator device processor 36, upon taking the original product video with the administrator interface 32. Then, in step 356 the original product video is received to the identification server 20 as reference image data of the original product over the communication network 100. The original product video may be, automatically or on request of the administrator, stored in step 358 to the original product database 25 in the one or more storage devices 28 in connection with the product profile of the new original product or separate from the product profile. Alternatively, the original product video may be stored in another storage device or the step 358 may be omitted.
The method of the present invention further comprises step 400 of automatically splitting, by utilizing instructions stored in a memory 24 and executed by a processor 22 of the identification server 20, the original product video into several separate original images. In one embodiment of the invention, the automatically splitting the original product video comprises step 402 of automatically splitting, by utilizing instructions stored in a memory 24 and executed by a processor 22 of the identification server 20, the original product video into video frames to create the several separate original images. In step 404 of
The method of the present invention comprises step 450 of automatically, or on request of the administrator, creating, by utilizing instructions stored in the memory 24 and executed by the processor 22 of the identification server 20, standardized separate original images by image pre-processing the several original images. The image pre-processing may further comprise step 452 in which the creating of standardized separate original images by image pre-processing the several original images may comprise cropping, scaling, flipping, rotating, filtering, utilising affine transformation or any other image processing method, by utilizing instructions stored in the memory 24 and executed by the processor 22 of the identification server 20, the several original images or several original video frames. The pre-processing may further comprise automatically selecting, by utilizing instructions stored in a memory 24 and executed by a processor 22 of the identification server 20, a number of the several separate original images to be image pre-processed into standardized separate original images. Thus, only part of the several separate original images may be pre-processed and sued and input data to the basic identification algorithm. This allows, utilizing only images fulfilling certain predetermined criteria.
In step 454 of
The method further comprises step 500 of providing a basic identification algorithm from an algorithm service 26. The basic identification algorithm may be provided to the identification server 20 from the algorithm service 26 or it may be formed in the algorithm service 26. The algorithm service 26 may comprise an artificial neural network or convolutional neural network.
The method also comprises step 550 of applying, by utilizing instructions stored in the memory 24 and executed by the processor 22 of the identification server 20, the standardized separate original images as input data to the basic identification algorithm provided from the algorithm service 26. This step 550 may be carried out by uploading or sending the standardized separate original images to the algorithm service 26 by utilizing instructions stored in the memory 24 and executed by the processor 22 of the identification server 20 and processing the standardized separate original images with the basic identification algorithm in the algorithm service 26. Alternatively, the step 550 may be carried out by identification server by utilizing instructions stored in the memory 24 and executed by the processor 22 of the identification server 20 and processing the standardized separate original images with the basic identification algorithm provided to the identification server 20 from the algorithm service 26. Accordingly, the basic identification algorithm is trained with the standardized separate original images to identify original products. Then, the reference identification algorithm is generated, by utilizing instructions of the algorithm service 26, from the input data, meaning the standardized separate original images, as the standardized separate original images are processed with the basic identification algorithm. The reference identification algorithm is specific to the new original product and created product profile.
When the reference identification algorithm is generated, then in step 600, the reference identification algorithm of the original product is connected, by utilizing instructions stored in the memory 24 and executed by the processor 22 of the identification server 20, to the product profile in the at least one original product database 25, stored in the one or more storage devices 28. Connecting the reference identification algorithm to the product profile may be carried out by storing the reference identification algorithm to the one or more storage devices 28 or to the algorithm service and connecting it to the product profile or by storing the reference algorithm to the product profile in the original product database 25. In this way, the original product data and the reference identification algorithm of the same original product are connected to each other for identification of products.
Accordingly, the steps 500 of providing the basic identification algorithm from the algorithm service 26 may comprise providing two or more basic feature identification algorithms or a basic identification algorithm comprising two or more feature sub-algorithms. As shown in
As shown in
Then the first, second and third feature identification algorithms may be or the combined reference identification algorithm may be connected to the product profile in the original product database 25 in the one or more storage devices 28. Furthermore, in a case of separate feature identification algorithms, the first, second and third feature identification algorithms may be combined to each other in step 560.
In one embodiment, the generating, by utilizing instructions of the algorithm service 26, the reference identification algorithm or the two or more feature reference identification algorithms comprises automatically generating, by utilizing instructions of the algorithm service 26, a shape reference identification algorithm for identifying shape 704 of the product 700. Additionally or alternatively, it may comprise automatically generating, by utilizing instructions of the algorithm service 26, a visual symbol reference identification algorithm for identifying visual symbols 706 on the product 700, and/or automatically generating, by utilizing instructions of the algorithm service 26, an optical character recognition reference identification algorithm for identifying written text 708 on the product 700.
The above described feature reference identification algorithms may divide identification process to successive steps or to parallel parts.
As mentioned above and shown in
As shown in
The sub-profiles or different reference identification algorithms of the sub-profiles may be connected to each other in the original product database 28, or the sub-profiles or different reference identification algorithms of the sub-profiles may be connected to the upper level product profile of the pharmaceutical product in the original product database 28. Alternatively, the sub-algorithms may be connect to each other or to the sub-profiles or the upper level product profile of the pharmaceutical product.
According to the above mentioned, the identification of authenticity of the pharmaceutical product may be carried out according to target image of any of the sides, surfaces or parts of the product having a sub-profile and reference identification sub-algorithm. Alternatively, the identification of authenticity of the pharmaceutical product may be carried out based on two or more target image of the sides, surfaces or parts of the product having a sub-profile and reference identification sub-algorithm. This may be carried out series or in parallel in the same manner described in connection with the feature reference identification algorithms. For example, the identification may be based on only outer packaging, inner packaging or the product itself. Alternatively, the identification may be based two or more of the outer packaging, inner packaging and the product itself.
According to the above mentioned, the sub-profiles or different reference identification algorithms of the original product may be used in combination of the feature reference identification algorithms, respectively. Thus, a very complex and detailed identification may be provided with the system and method of the present invention.
The image taking instructions may further instruct the user of the mobile device 10 to take the target image in a certain angle with respect to the product. These kinds of image taking instructions may be based on one or more accelerometers provided in the user device which give information relating to the angle of the user device with respect to the vertical and/or horizontal direction when the user device is stationary. This provides information about the angle in which the target image is taken with respect to the product when the product is placed on a horizontal surface.
In some embodiments, the user interface 12 may instruct in taking the target image. Accordingly, the user device 10 or the user interface 12 may instruct, by utilizing image taking instructions stored in the user device memory 15 and executed by the user device processor 16, in taking the target image. The user interface 12 in the mobile device 10 of the system 1 may be operable to instruct the user in taking the target image.
In one embodiment, the user device 10 or the user interface 12 may instruct, by utilizing image taking instructions stored in the user device memory 15 and executed by the user device processor 16, in taking the target image on a predetermined angle or in predetermined angles. The user device 10 may comprise one or more accelerometers for determining the position or angle of the user device 10 or the camera in the user device 10. Accordingly, the accelerometer may be operable to determining or calculating position or posture of the user device 10, for example by utilizing image taking instructions stored in the user device memory 15 and executed by the user device processor 16 of the user device 10. Accordingly, the user device 10 or the user interface 12 may instruct, by utilizing image taking instructions stored in the user device memory 15 and executed by the user device processor 16, in taking the target image on a predetermined angle or in predetermined angles based on the calculation or determination of the position or posture of the user device 10 with the one or more accelerometers.
In one embodiment, the user device 10 or the user interface 12 may instruct, by utilizing image taking instructions stored in the user device memory 15 and executed by the user device processor 16, placing the product on a horizontal plane or surface before taking the target image. Then, the user device 10 or the user interface 12 may instruct taking the target image in predetermined angle or angles.
According to the above mentioned, the target image may be instructed to be taken in one angle or turning the camera in the mobile device 10 in different angles during taking the target image.
In one embodiment, the user interface 12 in the user device 30 is operable in calculating, by utilizing image taking instructions stored in the user device memory 15 and executed by the user device processor 16 and the accelerometer, the position or posture of the user device 10 or the camera in the user device 10 during taking the target image. The calculated position or posture of the mobile device 10 or the camera in the user device during taking the target image may be send to the identification server 20 together with the target image, or separately. The calculated position or posture of the user device 10 or the camera in the user device may further be stored to the product profile. In this embodiment, the user device 10 or the user interface 12 may instruct, by utilizing image taking instructions stored in the user device memory 15 and executed by the user device processor 16, placing the product on a horizontal plane or surface before taking the target image. Then, the position or posture of the user device 10 or the camera in the user device is calculated during taking the target image.
In one embodiment, the user device 10 or the user interface 12 may instruct, by utilizing image taking instructions stored in the user device memory 15 and executed by the user device processor 16, in taking the target image on a predetermined angle or in predetermined angles based on the data provided to the product profile or reference identification algorithm. This data in the product profile or in the reference identification algorithm may be based on the image taking angle or angles of the original product video, as described above. Thus, the user device 10 or the user interface 12 may instruct to take the target in an angle corresponding the angle or angles in which the original product video is taken. Alternatively, the position or posture data of the target image and the original product video may be used, by utilizing instructions stored in the memory 24 and executed by the processor 22 of the identification server 20, to calculate or determine the authenticity of the product. Alternatively, the position or posture data of the target image and the original product video may be used, by utilizing instructions stored in the memory 24 and executed by the processor 22 of the identification server 20, to pre-process the separate original images and/or the target image. Furthermore, in some embodiment the position or posture data of the original product video may be used may be used in determining or calculating two- or three-dimensional shape or dimensions of the original product by utilizing instructions stored in the administrator device memory 35 and executed by the administrator device processor 36 and the accelerometer or by utilizing instructions stored in the memory 24 and executed by processor 22 of the identification server 20 and the accelerometer.
Accordingly, the standardization of the target mage may be carried out by automatically creating, by utilizing instructions stored in a memory 24 and executed by a processor 22 of the identification server 20, a standardized target image by image pre-processing the target image, or by creating on request of the user or automatically creating, by utilizing instructions stored in the user device memory 15 and executed by the user device processor 16 of the user device 10, a standardized target image by image pre-processing the target image in the user device.
It should be further noted, that.
The authenticity identification response in all the examples of this application does not need to be that the product of the target image is authentic or not authentic but the authenticity identification response can be for example an estimate about the authenticity. The authenticity identification response in all examples can be a visual response or an oral response, a text response or a numeral response, or any other type of response which can illustrate the authenticity or the estimate of the authenticity.
The invention has been described above with reference to the examples shown in the figures. However, the invention is in no way restricted to the above examples but may vary within the scope of the claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 20176056 | Nov 2017 | FI | national |
| 20185631 | Jul 2018 | FI | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/FI2018/050853 | 11/23/2018 | WO | 00 |
